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ABSTRACT 



This thesis presents the results of a mailed survey of 
all officers currently serving on active duty in the United 
States Marine Corps who possess a secondary Military Occupa- 
tional Specialty (MOS) of Operations Analyst (MOS code 9650) . 
The questionnaire used was designed to determine the usage 
frequency and the relative importance of each of 35 Operations 
Research (OR) techniques and application areas to these Marine 
officers during their most recent tours in 9650 billets. In 
addition, the results were classified according to individual 
service rank, function area and type of OR work performed and 
a series of nonparametric statistical tests was conducted to 
determine significant differences. Comments elicited by the 
survey were also analyzed and recommendations were made based 
upon the conclusions drawn. 
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BACKGROUND 



I . 



A. INTRODUCTION 

This thesis presents the results of a mailed survey of 
all officers currently serving on active duty in the United 
States Marine Corps who possess a secondary Military Occupa- 
tional Specialty (MOS) of Operations Analyst (MOS code 9650) . 
The questionnaire used was designed to determine the usage 
frequency and the relative importance of each of 35 Operations 
Research (OR) techniques and application areas to these Marine 
officers during their most recent tours in 9650 billets. In 
addition, the results were classified according to individual 
service rank, function area and type of OR work performed and 
a series of statistical tests was conducted to determine 
significant differences. Comments elicited by the survey 

r 

were also analyzed and recommendations were made based upon 
the conclusions drawn. 

B. HISTORY OF OPERATIONS ANALYSIS IN THE MARINE CORPS 

The Marine Corps initiated an "in-house" operations analy- 
sis capability in 1957, when a civil service billet was estab- 
lished at Marine Corps Headquarters (HQMC) in Washington, 

D.C. [Ref. 1] . Subsequently, the Commandant of the Marine 
Corps decided that there existed a need for Marines educated 
at the graduate level in the field of Operations Research/ 
Systems Analysis (OR/SA) so, in 1958, the first Marine officer 
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was assigned to the Naval Postgraduate School (NPS) to study 
their Operations Research curriculum. During the next few 
years only about four or five Operations Analyst billets were 
identified by the Marine Corps, consequently, the input of 
Marines to the NPS program was small, averaging only about 
two per annum through 1963. Although limited operations analy- 
sis support was available to the Marine Corps through the 
Navy's CNO Operations Evaluation Group, the Marines did not 
possess a significant analysis capability until 1965, when 
the Marine Corps Operations Analysis Group was established as 
a separate division at the Center for Naval Analysis, a pri- 
vate, non-profit organization under contract to the Navy and 
administered by the University of Rochester. That same year, 
the first Fleet Marine Force (FMF) operations analysis sec- 
tion was formed at FMF Pacific. 

During the middle sixties the requirements for Marine 
Operations Analysts accelerated rapidly due to an increased 
recognition of the value of analytical problem solving. 
Correspondingly, the annual personnel input to the OR/SA 
curriculum at postgraduate school increased until the current 
level of about 10 officers per year was reached. Although the 
great majority of these officers attend the Naval Postgraduate 
School at Monterey, California, a few also attend civilian 
universities. Presently, there are 19 Marine officers enrolled 
in the OR/SA curriculum at the Naval Postgraduate School and 
2 enrolled at civilian schools. 
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These personnel, once educated, will be assigned the 
Military Occupational Specialty code 9650 and given the title 
Operations Analyst. The Marine Corps MOS Manual describes 
the duties and tasks which these people should be capable of 
completing as follows : 

Duties and Tasks : Participates in the following: 

construction of models, manual or computer, for 
analysis or comparison of military activities, 
operations, weapons systems, and force structures; 
the gathering of empirical data used in support of 
analysis, comparison, or war gaming of military 
activities, weapons systems and plans; cost effective- 
ness studies to analyze weapons systems, operational 
concepts, and resource allocation. Conducts sta- 
tistical evaluations and comparisons of the results 
of such tests and writes technical reports on results. 
Directs, supervises, conducts, or participates in war 
gaming or simulation activities, including preparation 
of rules or flow charts, conduct of game or simulation, 
assessment of the game moves, of simulation steps, 
evaluation of the results, and determination of appro- 
priate measures of effectiveness. Performs technical 
liaison between military organizations and activities 
engaged in analytical or evaluation work and techni- 
cal assistance [for] contractors supporting the 
military activity. Performs any or all of the above 
duties and tasks at research and development activi- 
ties, management activities, or with operational 
units of the Fleet Marine Force. [Ref. 2] 

In less sophisticated terms, a Marine Operations Analyst 

is an individual capable of performing operations analysis, 

where such analysis, as defined by the Marine Corps, is "the 

application of scientific methods to perform analysis directed 

toward improving the operational effectiveness of organizations 

and the operating efficiency of organized man-machine systems 

in achieving their objectives" [Ref. 3:p. 1]. While the term 

"operations analysis" and "operations research" are synonymous, 

the term "systems analysis" carries a slightly different 
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connotation, being "the application of scientific methods to 
assist a decision maker to choose among alternatives by a 
systematic investigation of the relevant costs, risk and meaning- 
ful effectiveness of alternative systems, equipments or 
organizations" [Ref. 3:p. 2]. 

Operations analysis is used to address technical, tactical 
problems while systems analysis often addresses the more 
loosely defined strategic problems. The essence of an Opera- 
tions Analyst's work is to provide a decision maker with an 
analytical method for choosing among alternative procedures, 
allocations and/or courses of action. 

We can gain a better idea of operations analysis in the 
Marine Corps by reviewing how analysts are trained. 

C. MARINE CORPS SPECIAL EDUCATION 

To support the educational requirement for trained Opera- 
tions Analysts, the Marine Corps has two advanced education 
programs: the Special Education Program (SEP) [Ref. 4] and 

the Advanced Degree Program (ADP) [Ref. 5]. ‘ 

The Special Education Program allows voluntary selectees 
to attend either the Naval Postgraduate School or one of a 
limited number of specified civilian universities in order 
to study for an advanced degree in a number of selected fields, 
including OR/SA. The Marine Corps pays all tuition and re- 
quired academic fees as well as providing normal pay and 
allowances while the officer is attending school. 

The Advanced Degree Program was established to augment 
the SEP by providing an additional means for career officers 
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to obtain a graduate education in order to qualify for SEP 
billets. Through this program, officers may voluntarily 
apply to civilian universities acceptable to the Marine Corps 
to study in a field in which the Corps has realized a per- 
sonnel shortage. During the course of attendance, each offi- 
cer continues to receive his or her normal pay and allowances 
from the Marine Corps but must bear the cost of tuition and 
textbooks . 

Officers applying for either program must agree not to 
submit a resignation nor to request separation while partici- 
pating in the program and must further agree to remain on active 
duty, after completion of their education or upon separation 
from the program, for a period of 3 to 4 years (depending upon 
the length of their course of instruction) . Upon successful 
completion of one of these programs, the officers are assigned 
a secondary MOS and are sent to SEP billets requiring the use 
of their newly acquired skills. (The only exceptions to this 
immediate SEP assignment policy are officers who are scheduled 
for an overseas assignment or officers who have studied in 
a field in which no billets are currently available) . In any 
event, an officer with a SEP MOS can expect to be required to 
serve alternating tours in SEP billets thereafter. 

D. SPECIAL EDUCATION PROGRAM BILLETS 

The Marine Corps currently (1980) has 414 billets desig- 
nated as Special Education Program (SEP) billets, the number 
of billets authorized at any time being the result of an 
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iterative justification process. Recommendations for estab- 
lishment of SEP billets can be made to the Deputy Chief of 
Staff for Manpower by 

1) Marine Corps field commanders (Divisions, Wings, etc.) 

2) SEP discipline sponsors 

3) Table of Organization (T/0) sponsors 

SEP discipline sponsors are the HQMC staff sections desig- 
nated as the "in-house" expert on their particular discipline. 
(The SEP discipline sponsor for OR/SA is the Deputy Chief of 
Staff for Research, Development and Studies.) The T/0 sponsor 
is the HQMC staff section with administrative responsibility 
for the Table of Organization which includes the recommended 
billet. 

Specific requirements and justification for the establish- 
ment of a new billet must be submitted in the form of a com- 
pleted Billet Education Evaluation Certificate. In order for 
a billet to be identified as requiring graduate level educa- 
tion, it must be determined that such a level is either 
"Necessary" or "Desirable" for the proper functioning of the 
billet incumbent. The classification of "Necessary" is assigned 
only to billets meeting one of the following criteria: [Ref. 6: 

p. 2] 

1) Billets which are required by law or DOD policy to be 
filled by individuals possessing graduate level 
education in a specific field of study. 

2) Billets in which the primary duties of the incumbent 
cannot be satisfactorily performed except by 
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individuals possessing qualifications which can be 
acquired only through graduate education in a relevant 
field of study. 

3) Billets which must be filled by individuals who are 
required to exert direct technical supervision over 
military or civilian billets with graduate education 
prerequisites . 

The classification of "Desirable" is assigned to billets not 
meeting any of the above criteria, but which can be most 
effectively filled by individuals who have had graduate training. 

In the billet approval process the discipline sponsor has 
the major responsibility for determining if recommended billets 
actually require graduate education. In addition, the sponsor 
is required to conduct an annual review of all current billet 
educational requirements to ensure that each educational level 
is continually warrented. 

There are, presently, 53 SEP billets specified as being 
Operations Analyst billets. (The number of Marine Operations 
Analysts currently on active duty is 77, well below the 127 
necessary in order to meet the Marine Corps desired level of 
2.4 trained officers for each billet.) The Marine Corps 
Operations Analysis Group located at the Center for Naval 
Analysis currently represents the largest single OR/SA capa- 
bility in the Marine Corps. This organization, comprised 
of civilian and Marine Operations Analysts and support per- 
sonnel, provides a detachment of analysts to the Marine Corps 
Development and Education Center (MCDEC) and one analyst to 
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each of the FMF Headquarters. Additionally, smaller opera- 
tions analysis sections consisting of military and/or civil 
service Operations Analysts have been established as well as 
certain individual billets. The majority of these sections 
and billets are currently located at HQMC and MCDEC although 
there are also individuals located with joint commands such 
as the Department of Defense, the Joint Chiefs of Staff and 
the Defense Communications Agency. 

The majority of Operations Analyst billets being spread 
throughout the Marine Corps rather than being pooled at a 
single location reflects the current Marine Corps policy that 
Operations Analysts should provide direct support to individual 
commanders and staff section heads. In fact, "to employ 
analysts in a substructure below this level tends to reduce 
their effectiveness" [Ref. 3:p. 5], 

E. PROBLEM STATEMENT 

Upon completion of interviews with several persons pre- 
sently assigned to MOS 9650 SEP billets, the author concluded 
that the official Marine Corps explanation of the tasks and 
duties assigned to these personnel does not always reflect 
the actual functions of their billets. In addition, discussions 
with students enrolled in the OR/SA curriculum at NPS revealed 
an unclear perception of the real responsibilities and func- 
tions of Mhr.ine Operations Analysts "in the field". This re- 
sults from the lack of a published, clear definition of the 
actual roles played by Marine Operations Analysts and an 
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explanation of the techniques that they employ. What tasks 
are Marine Operations Analysts actually being required to 
complete? Are they being taught the techniques that they will 
be obliged to understand in order to complete their tasks? 

A method of answering these questions is to ask the officers 
currently assigned the 9650 secondary MOS what tasks they 
have been required to complete and what techniques they used. 
The answers from such a survey can then be utilized in the 
evaluation and possible modification of the curriculum em- 
ployed in educating potential Marine Operations Analysts at 
the Naval Postgraduate School. Additionally, a survey of OR/ 

SA techniques currently in use by Marine analysts may provide 
a basis for developing a program of elective courses that might 
be recommended to future Marine Corps students at NPS. 

The purpose of this thesis was to conduct a survey of the 
Operations Research techniques currently being used by Marine 
9650's and, through statistical inference and comparative 
analysis, make recommendations for improving the training 
and utilization of Marine Operations Analysts. 

F. PREVIOUS STUDIES 

Studies of OR technique usage during the last ten years 
include surveys of both corporate and military usage. In 
1977, Ledbetter and Cox surveyed the 500 largest U.S. indus- 
trial firms as listed by Fortune magazine in 1975 [Ref. 7] . 
Analyzing their 176 returns with respect to the use of OR 
techniques and the specific areas of OR application, they 
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found that a great majority of the responding firms used 
regression analysis to at least some degree, while a nearly 
like number used simulation (in production) and linear pro- 
gramming (see Table I) . Simulation and linear programming 
reflected the greatest breadth of usage over a wide range of 
application areas while regression analysis showed the highest 
ranking for across-the-board degree of usage within the 
firms . 



TABLE I 

CORPORATE OR TECHNIQUE USAGE 
Technique 

Regression Analysis 
Simulation 
Linear Programming 
Queueing Theory 
Network Models 
Dynamic Programming 
Game Theory 



(LEDBETTER AND COX) 

% of Firms Reporting Usage 

90.5 

88.6 

84.6 

63.4 
60.9 

46.4 
40.3 



In their study of 137 Case Western Reserve University 
alumni, Rasmussen and George asked the respondents to report 
the value to them during their careers of various theoretical 
and methodological subject areas related to the Operations 
Research field [Ref. 8] . The following order of value was 
reported by respondents possessing a Master of Science degree: 
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Statistical Methods 
Systems Analysis 
Forecasting 
Information Systems 
Simulation 
Decision Theory 
Linear Programming 
Scheduling Theory 
Production Management 
Inventory Theory 
Stochastic Processes 
Non-linear Programming 
Integer Programming 
Dynamic Programming 
Queueing Theory 

Other techniques and related subject areas considered to 
be important by those answering the survey included: Networks 

Graphs, Reliability Theory, Modeling, Case Studies, Problem 
Formulation, Computing, Cost/Benefit Analysis, Structure of 
Organizations, Philosophy of Management, Communications and 
"the organization survival of the Operations Research prac- 
titioner". Also reflected in this study were the inferences 
that Ph.D. respondents felt that the Mathematical Programming 
areas were more useful than the M.S. respondents did and that 
Ph.D. respondents reported a higher level of receptivity to 
OR techniques by their employers than did the M.S. respondents 
In research conducted in 1977, Thomas and DaCosta [Ref. 9] 
compared their survey of 150 California firms and institutions 
and "Fortune 500" corporations with surveys that had been con- 
ducted by Hovey and Wagner [Ref. 10] in 1958, Schumacher and 
Smith [Ref. 11] in 1964, and Turban [Ref. 12] in 1969. Their 
findings on usage of OR techniques, as shown in Table II, were 
similar to those of Ledbetter and Cox. In addition, they 
found that, when the users were queried as to the types of 
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TABLE II 



CORPORATE OR TECHNIQUE USAGE (THOMAS AND PACOS TA) 



Technique 

Statistical Analysis 
Simulation 
Linear Programming 
PERT/CPM 

Inventory Theory 
Queueing Theory 
Non-linear Programming 
Heuristic Programming 
Bayesian Decision Analysis 
Dynamic Programming 
Risk Analysis 
Integer and Mixed Programming 
Delphi 

Financial Methods 



% of Firms Reporting Usage 
93 
84 
79 
70 
57 
45 
36 
34 
32 
27 
3 
2 
1 
1 



difficulties that are encountered in attempting to implement 
OR work, the most frequently mentioned difficulties were 
user commitment, user understanding and organizational resistance 
to change. Specifically, they felt that "if the manager who 
must use the proffered solution does not understand its basis 
there will be friction and difficulties" [Ref. 9:p. 108]. 

All of the above-noted studies were related to private 
industry which has more easily-measured overall objectives 
such as profit and profit contribution. This might well cause 
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corporate survey results to be far different from those that 
would be obtained from an inquiry into the usage of OR tech- 
niques by military personnel. 

In 1978, Obert conducted a telephone survey of ninety 
percent (328) of the Operations Research and Systems Analysis 
positions within the Army [Ref. 13] . His figures for fre- 
quency of OR technique usage for individuals characterized as 
holding OR/SA engineer billets are shown in Table III. One 
should bear in mind that the principal difference between this 
survey and most of the ones previously mentioned is that in 
this survey individuals were asked to report frequency of 
usage of OR techniques while, in the corporate surveys, 
organizations were asked to report usage frequencies. Obert 
found that almost ninety percent of his respondents character- 
ized their duties as either doing original OR/SA research, 
correlating the work of other researchers or reviewing OR/SA 
work. Additionally, he found that while over eighty percent of 
his respondents worked with established computer programs, the 
majority did not do their own computer programming (FORTRAN 
being the programming language most frequently employed by 
the program-writing minority) . Finally, it was determined 
that, while the overall results for OR/SA engineers showed that 
almost three-fourths of the respondents said that they spent 
one-half or less of their time using quantitative analysis 
or OR/SA techniques compared to their other daily duties, 
Captains and Majors, apparently, spent more of their time 
doing analysis than did Lieutenant Colonels and Colonels. 
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TABLE III 



OR TECHNIQUE USAGE BY ARMY OR/SA ENGINEERS 



Technique % of 


Individuals 


Parametric Statistics 


98 


.5 


Probability 


96 


.5 


Formal Data Analysis 


93 


.9 


Measures of Effectiveness 


78 


.8 


Simulation Models and Applications 


73 


.8 


Networks, Flows and Graphs 


65 


. 6 


Combat Models 


64 


. 6 


Decision and Risk Analysis 


64 


.2 


War Games 


63 


.1 


Experimental Design 


54 


.1 


Nonparametric Statistics 


48 


.5 


Human Factors 


45 


.4 


Stochastic Models 


44 


.9 


Learning Curves 


44 


.5 


Cost Estimating 


41 


.9 


Linear Algebra 


41 


.4 


Linear Programming 


37 


.9 


Calculus 


32 


.8 


Economics and Economic Analysis 


25 


.7 


Manpower Models 


24 


. 6 


Accounting and Financial Applications 


23 


.2 


Dynamic and Non-linear Programming 


21 


.7 


Cost Estimating Relationships 


19 


.2 


Economic Input-Output Models 


8 


.1 
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The following chapters explain the planning, conduct and 
results of the work undertaken by the author in an attempt 
to ascertain the frequency of usage of OR techniques in the 
Marine Corps. Chapter II describes the experimental methodology 
developed to obtain measurable results from a survey of Marine 
Operations Analysts. These results are presented and analyzed 
in Chapter III. Numerical data obtained from the question- 
naires are categorized into several groups and the statisti- 
cal tests that were conducted are described. Comments made 
by the respondents are grouped according to theme and sub- 
jectively discussed. Finally, in Chapter V, conclusions 
are drawn from the results of the analysis and recommendations 
for the improved utilization of 9650 's are made. 
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II. METHODOLOGY AND CONDUCT OF THE EXPERIMENT 



Since the experiment population was well defined (all 
Marine personnel holding a MOS of 9650 and currently on active 
duty) and equally easy to locate, it was decided that a mailed 
questionnaire, as opposed to a telephone survey, would be the 
most appropriate method of conducting the survey. This would 
provide a controlled, unbiased means of obtaining answers 
while allowing individuals the time necessary to think about 
each question before responding. Since a mailed questionnaire 
might result in a lower response rate than could be obtained 
via personal contact, it was decided that, upon reaching a 
return rate of 75%, the remaining individuals who had not 
responded would be contacted by telephone to determine if they 
required additional information or assistance. A final response 
rate goal of 90% was deemed to be reasonable since it was felt 
that at least 10% of the population might not be able to be 
reached due to being deployed overseas, enroute to a new duty 
station, on leave, etc. 

The questionnaire was designed so that each question would 
obtain replies which could be summarized in a fashion capable 
of answering questions implied by the thesis objectives (see 
Appendix A for questionnaire format) . The first four questions 
were directly related to the respondents' billets, requesting 
the billet title, location and general function area as well 
as the service rank of the individual and the type of OR work 
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that the billet required him to do the most. The answers 
obtained from these questions would be used to group the 
returns for detailed study during their analysis. Questions 
5 and 6 were related to computer programming and the use of 
preprogrammed computer packages, inquiring as to the types 
of computer languages used, if any, and the reasons for their 
usage. In an attempt to secure some measure of the usage 
frequency and importance of OR techniques to the practicing 
analysts, questions 7 and 8 were developed, allowing the 
respondents to rate each subject according to their personal 
experiences. Question 9 was added as an attempt to include 
in the analysis any subject (not included in questions 7 and 
8) that the respondents used frequently or felt to be impor- 
tant to them. Question 10 was an attempt to ascertain how 
much of their work time analysts are spending on OR labors. 
Question 11 would help to determine the acceptance level of 
OR work "in the real world", while responses obtained from 
question 12 would result in subjective recommendations for 
the better utilization of Marine Operations Analysts. 

Recognizing that the answers obtained would be clearly 
of a subjective nature and that the form of the underlying 
distribution of the answers would be unknown, it was decided 
that nonparame trie (i.e., distribution-free) methods of statis- 
tical analysis would be appropriate for analyzing the returns. 
Parametric methods require that more assumptions be made than 
do nonparame trie methods, consequently nonparametric statistical 
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tests are generally less powerful than parametric statistical 
tests . 

This power-deficiency can be corrected when using non- 
parametric methods, however, by drawing a larger sample size 
from the population [Ref. 14 :p. 21] (hence, the imperative of 
the return rate goal of 90%) . One assumption that must be 
made, though, whether using parametric or nonparametric 
methods is that the observations are independent. Additionally, 
nonparametric methods as used in this study require that all 
questions intended to be analyzed be designed so that answers 
are at least based on an ordinal scale and that all observa- 
tions are drawn from an underlying continuous distribution. 

The strategy for the data analysis was first to determine 
the overall frequency of usage and perceived importance of a 
number of OR techniques and application areas and then to 
determine if there were differences according to frequency 
of usage or perceived importance. One might expect that there 
would be differences and this, in fact, was borne out by the 
results. A more difficult question to answer, however, is 
what are the possible causes of the differences. The data 
were grouped according to the service rank of the individuals, 
the functional area related to their billets and the type of 
OR work that their billets required them to do the most and 
statistical tests were conducted to determine possible causes 
for differences and relationships between the groups. These 
tests, the conclusions drawn from them and various recommenda- 
tions are presented in the chapters that follow. 
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III. DATA ANALYSIS 



A copy of the survey questionnaire contained in Appendix 
A was sent to each of the 77 Marine officers designated as 
Operations Analysts (Military Occupational Specialty 9650) who 
are currently on active duty. Individuals were asked to base 
all of their answers on the most recent MOS 9650 billet in 
which they had served. Those who had never served in a MOS 
9650 billet were asked to state so and return the unanswered 
questionnaire. A total of 72 persons returned their question- 
naires for a return rate of 93.5%. Six of these people stated 
that they had never held a 9650 billet, leaving a usable sam- 
ple of 66 respondents for analysis (i.e., a 83.3% usable re- 
turn rate) . 

A. RANK 

Table IV contains frequency-of-answer data relating indi- 
vidual service rank. The reader is reminded that the tabulations 
do not reflect the current status of the population, but. 



TABLE IV 

SERVICE RANK DURING MOST RECENT 9650 BILLET 



Rank 


Raw Score 


(Percentage) 


Captain 


17 


(25.8) 


Major 


35 


(53.0) 


Lieutenant Colonel 


12 


(18.2) 


Colonel 


_2 


( 3.0) 


Total 


66 


(100.0) 
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rather, the status of each individual during his most recent 
tour in a 9650 billet. As an illustration: 40 of the 77 

officers sent questionnaires currently hold the rank of Lieu- 
tenant Colonel but only 12 officers of the 66 respondents re- 
ported that they held that rank during their most recent 9650 
billet assignment. As can be seen, a majority of the respon- 
dents said that they had held their billets as Majors. 

B. FUNCTION AREA 

Respondents were fairly evenly spread among functional 
areas of employment (see Table V) . Eight individuals reported 
that their assignments were related to a combination of the 
listed function areas while an equal number said that their 
jobs were associated with a different field of work than those 



TABLE V 

FUNCTIONAL AREA 



Functional Area 


Raw Score 


(Percentage 


Manpower 


11 


(16.7) 


Logistics 


8 


(12.1) 


Finance 


1 


(1.5) 


Operations and Plans 


7 


(10.6) 


Test and Evaluation 


14 


(21.2) 


Aviation 


6 


(9.1) 


Instruction 


3 


(4.5) 


Combination 


8 


(12.1) 


Other 


_8 


(12.1) 


Total 


66 


(100.0) 
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listed (e.g. , system development, research and development 
[5] , studies and long-range planning) . 

C. TYPE OF OR WORK DONE 

As Table VI indicates, one-third specified that they did 
original OR work most of the time. A nearly like number 
(34.8%) reported that most of their OR work was done review- 
ing work completed outside of their sections. Two questions 
of interest: Does the most frequent type of OR work done 

vary by rank and does the most frequent type of OR work done 
vary by function area? 



TABLE VI 

TYPE OF OR WORK DONE 



Type of Work 


Raw Score 


(Percentage) 


Original 


22 


(33.3) 


Correlate 


4 


(6.1) 


Review 


23 


(34.8) 


Instructor 


3 


(4.5) 


Combination 


6 


(9.1) 


Other 


3 


(4.5) 


None 


_5 


(7.6) 


Total 


66 


(100.0) 


TYPE OF WORK DONE BY RANK OF 
To answer the first question 


OFFICER 
, the null 


hypothesis that 



there is no difference in the frequency distribution of type 
of OR work done between the more junior ranks of Captain and 
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Major and the more senior ranks of Lieutenant Colonel and 

Colonel was tested. The associated frequency distributions 

2 

are shown in Table VII. A Chi-square (x ) test 

indicates that there is not a significant difference between 

TABLE VII 

SERVICE RANK VERSUS TYPE OF OR WORK DONE 

Captains/ Lieutenant Colonels/ Total 



Original 


Majors 

20 


Colonels 

2 


22 


Correlate 


4 


0 


4 


Review 


15 


8 


23 


Instructor 


2 


1 


3 


Combination 


6 


0 


6 


Other 


1 


2 


3 


None 


4 


1 


5 




_ri “ " 


1 


— 


Total 


52 


14 


66 


X 2 (6) = 

P Value 


11.138 
= 0.084 







junior and senior officers based upon the type of OR work per- 
formed (P = 0.084). (The P value is the smallest value of 
type I error probability for which there would be a rejection of the 
null hypothesis. It was decided by the author that a critical 
level of 0.05 would be utilized for the rejection of any null 
hypothesis in this study) . 

It should be noted that, in only two of the categories 
listed, the expected frequencies are greater than five. Some 
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2 

authors feel that when the degrees of freedom of a x test 

2 

are greater than one the x test should be used only if fewer 
than twenty percent of the category cells have an expected 
frequency of less than five and if none has an expected 
frequency of less than one [Ref. 14 :p. 110] . However, 

Lancaster notes that 

it used to be feared by authors that the test would 
not be accurate if carried out when any class expec- 
tation was less than 10. These fears have proved 
unjustified by the investigations of authors using 
simulation methods or complete enumeration of all 
possibilities in given discrete distributions. [Ref. 15 :p. 175] 

While Lancaster does go on to concede that it is "probably 

desirable" to have all expectations greater than one, he does 

not commit himself to making any concrete ruling on the matter. 

2 

As a check on the above test, a x test was also conducted 
with a collapsed contingency table comprised of row categories: 
"Original", "Review" and "Other". A P value of 0.105 was 
obtained, reinforcing the initial failure to reject the null 
hypothesis . 

E. TYPE OF WORK DONE BY FUNCTIONAL AREA 

In addressing the question of the relationship of the type 

2 

of most frequent OR work done and function area, a x test of 
independence was again used. However, when a contingency table 
was set up utilizing all of the categories in questions 3 and 
4 of the questionnaire, it was found that 63% of the cell ex- 
pected frequencies were less than one. Since this seemed to 
be stretching the rule-of-thumb for cell minimum expected 
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frequencies a bit too far, the categories were collapsed to 
those shown in Table VIII. The hypothesis that there is no 
difference between individuals employed in each of the listed 
function areas based upon the type of OR work their billets 
require them to do the most was rejected (P = 0.0005) . 



TABLE VIII 

FUNCTIONAL AREA VERSUS TYPE OF OR WORK DONE 



Original 

Review 

Correlate , 
Instructor , 
Combination , 
Other 



cn 

o 
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$ 
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ttj 



I 




r— I 

£ 



4 2 1 



5 3 7 0 



22 



4 4 3 



6 303 23 



2 0 2 3 



0 18 



16 



Total 



10 6 6 14 6 8 11 61 



X 2 (12) = 34.92 
P Value = .0005 



F. COMPUTER PROGRAMMING 

Only 39.4% of the respondents reported that their billet 
assignments required them to do their own computer programming. 
It was interesting to note that, while 44.2% of those who had 
been Captains or Majors during their most recent 9650 tour re- 
ported being required to do their own programming, only 21.4% 
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of those who had been Lieutenant Colonels or Colonels reported 
such. Table IX lists the reported frequency of programming 
language use. Fortran is by far the language most commonly 
used by the surveyed analysts, familiarity and availability 
being the most often reported reasons for its usage. In re- 
gard to all of the other listed languages, availability and 
scientific suitability of the language were the most often 
reported reasons for usage. 



TABLE IX 

TYPE OF COMPUTER PROGRAMMING LANGUAGE USED 



Language 


Frequency of Answer 


(Percentage ) 


Fortran 


19 


(38.8) 


Basic 


9 


(18.4) 


Mark IV 


6 


(12.2) 


APL 


5 


(10.2) 


GPSS 


3 


(6.1) . 


SPSS 


3 


(6.1) 


Cobol 


3 


(6.1) 


NIPS 


_1 


(2.0) 


Total 


49 


(100.0) 



A clear majority (66.7%) of the respondents reported inter- 
acting with established computer programs but, this time, a 
higher percentage of Lieutenant Colonels and Colonels (71.4%) 
reported using established programs than did Captains and 
Majors (65.4%) . 
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G. FREQUENCY OF USE OF OR TECHNIQUES 

In question 7 the respondents were asked to indicate the 
frequency of usage of each of 35 OR techniques and application 
areas by specifying "Never", "Sometimes" or "Always". Appen- 
dix B is comprised of frequency tables for the answers obtained 
while cumulative frequencies for each subject are presented 
in bar graphs in Figure 1. Each of the surveyed techniques 
and application areas is listed on the horizontal axes of the 
graphs while the vertical axes represent the percentage of 
individuals responding. The cross-hatched section of each 
column indicates the percentage of personnel reporting that 
they use the techniques "Always" while the clear enclosed sec- 
tion reflects the percentage reporting that they use them 
"Sometimes". As an example, in Figure 1 the graph shows that 
21% of the 66 respondents stated that they used Probability 
Theory "Always" and 56% said that they used it "Sometimes". 

Thus, a total of 77% of the sample reported using Probability 
Theory at least sometimes while the remaining 23% of the graph 
is left blank, representing those who responded that they never 
use that particular subject. 

Parametric Data Analysis and Statistical Inference, Cost 
Effectiveness, Networks and Probability Theory received the 
greatest number of responses for "Sometimes" and "Always" while, 
surprisingly, all of the mathematical programming techniques, 
including Linear Programming, received few usage responses. 
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Figure 1. FREQUENCY OF USE OF OR TECHNIQUES 



100 
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Econ Bayesian Capital inventory Non- Dynamic Heuristic Int & 
Models Decision Budget Theory Linear Prog Prog Mixed 
Analysis Prog Prog 



H. IMPORTANCE OF OR TECHNIQUES 

In question 8 of the survey individuals were asked to re- 
cord the relative importance to them on a scale of 1 = Extremely 
Important to 5 = Not Important of having a working knowledge 
of the 35 listed OR techniques and application areas in meeting 
the daily requirements of their 9650 billets. The complete 
listing of answer frequencies is contained in Appendix B while 
Figure 2 includes bar graphs representing the cumulative re- 
sponse frequencies for each subject. The surveyed techniques 
and application areas comprise the horizontal axes of the 
graphs and the percentage of individuals responding are repre- 
sented by the vertical axes. The clear enclosed columns repre- 
sent the cumulative percentage of individuals who marked a "1", 
"2" or "3" (extremely important to average importance) next to 
each subject while the remaining portion left blank represents 
the percentage of individuals who indicated a "4" or a "5" 
(little or no importance) as their answer. For example, Figure 
2 shows that a total of 77% of the 66 billet-holders responding 
said that they felt that Probability Theory fell in the "1", 

"2" or "3" category of their individual importance scales while 
23% felt that it held a lower position on their scales ("4" or 
"5") . It can be seen that, as was the case in usage frequency. 
Parametric Data Analysis, Probability Theory, Cost Effectiveness 
and Networks hold the highest positions in an ordering of 
importance based upon answers of "1", "2" and "3" while the 
advanced mathematical programming methods (Non-linear, Heuris- 
tic, Dynamic, Integer and Mixed Programming) hold the lowest. 
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Figure 2. IMPORTANCE OF OR TECHNIQUES 
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Figure 2 (CONTINUED) 



I. IMPORTANCE AND USAGE FREQUENCY COMPARISONS 

If we take responses "1 " , "2" and " 3 " from question 8 to mean Ex- 
tremely Important, Very Important and Average Importance, re- 
spectively, we can order the listed techniques by importance. 
Combining "Sometimes" and "Always" from question 7 also allows 
us to order these techniques by frequency of usage. We can, 
then, compare order based upon frequency of usage ("Sometimes" 
and "Always") (Table X) with order based upon frequency of 
importance (average or greater importance) (Table XI). It 
appears that there is a strong correlation between utilization 
and perceived importance. The only individual subjects that 
do not fall within a reasonable number of places on both lists 
are Equipment Replacement and Transportation (which are higher 
on the usage list than on the importance list) and Linear 
Algebra and Policy Analysis (which are higher on the importance 
list) . This may indicate that, while Equipment Replacement and 
Transportation are applied quite often, they do not seem to 
be very important to the using individuals. Additionally, 
it may be that, while Linear Algebra and Policy Analysis may 
not be used very often, when it does become necessary to use 
them, it is important to know how to use them. Another possible 
explanation for the subjects which fall lower on the usage list 
than on the importance list is the imperative of knowing where 
a technique does not fit. The implication is that, knowledge 
of a certain technique and knowing when not to use it can be 
just as important as knowing when to use it. 
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TABLE X 



ORDER BY FREQUENCY OF USAGE 



Subject 

Data Analysis (Parametric) 

Cost Effectiveness 
Networks, Flows and Graphs 
Probability Theory 
Cost Estimation 
Design, Conduct and Evaluation 
of Experiments 

Data Analysis (Nanparametric) 

Reliability 

Computer Prograntning 

Human Factors 

Cost Models 

Equipment Replacement 

Economics 

Calculus 

Maintenance and Repair 
Combat Analysis 
Risk Analysis 
Combat Models 
Manpower Models 
Transportation 
Linear Algebra 
Queueing Theory and 
Stochastic Models 
Gaming Models 
Forecasting 
Input-Output Models 
Linear Programming 
Policy Analysis 
Econometric Models 
Bayesian Decision Analysis 
Capital Budgeting 
Inventory Theory 
Non- linear Programming 
Dynamic Programming 
Heuristic Programming 
Integer and Mixed Programming 



Percentage (Sometimes + Always) 

81 

80 

79 

77 

73 

70 

67 

62 

58 

53 

53 

52 

51 

51 

51 

49 

45 

44 

42 

42 

40 

40 

38 

37 

37 

35 

32 

28 

27 

26 

21 

15 

14 

11 

8 
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TABLE XI 



ORDER BY IMPORTANCE 



Subject 



Percentage (Categories 1 + 2+3) 



Data Analysis (Parametric) 


77 


Probability Theory 


76 


Cost Effectiveness 


74 


Networks, Flews and Graphs 


73 


Canputer Programming 


70 


Data Analysis (Ncnparametric) 


68 


Reliability 


67 


Design, Conduct and Evaluation 




of Experiments 


64 


Cost Estimation 


64 


Cost Models 


61 


Ec enemies 


59 


Calculus 


58 


Linear Algebra 


58 


Human Factors 


56 


Risk Analysis 


55 


Combat Analysis 


50 


Forecasting 


47 


Policy Analysis 


47 


Maintenance and Repair 


47 


Manpcwer Models 


45 


Gaming Models 


45 


Linear Programming 


44 


Equipment Replacement 


44 


Queueing Theory and 




Stochastic Models 


44 


Combat Models 


41 


Econometric Models 


38 


Capital Bixlgeting 


38 


Transportation 


38 


Input-Output Models 


35 


Bayesian Decision Analysis 


33 


Inventory Theory 


29 


Non-linear Programming 


21 


Integer and Mixed 




Programming 


18 


Heuristic Programming 


17 


Dynamic Programming 


15 
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The association between importance and utilization of OR 
techniques was measured using the Spearman rank correlation 
coefficient. For the two lists in Tables X and XI a rank 
correlation coefficient of +0.918 was computed, indicating a 
highly positive correlation. Testing the null hypothesis that 
there is no relation in the reported order of usage frequency 
and the reported order of importance frequency of these OR 
subjects resulted in a P value of nearly zero, indicating 
that, at almost any level of significance, the null hypothesis 
of no relationship would be rejected. 

J. USAGE BY OFFICER RANK 

A comparison of usage frequencies was also analyzed by 
seniority of rank. Appendix C contains frequency tables based 
on reported usage of subjects for Captains /Majors and Lieutenant 
Colonels/Colonels. Figure 3 shows comparisons between Captains/ 
Majors and Lieutenant Colonels/Colonels for reported usage 
frequencies for each of the listed OR techniques and applica- 
tion areas. The cross-hatched column over each subject indi- 
cates the percentage of Captains and Majors who reported using 
that subject either "Sometimes" or "Always" and the clear en- 
closed column indicates the percentage of Lieutenant Colonels 
and Colonels who reported using each subject "Sometimes" or 
"Always" . 

To determine the greater frequency of usage for individual 
techniques, two methods were utilized, the first method being 
an absolute difference technique. Because of the small number 
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SERVICE RANK USAGE COMPARISON 
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Figure 3. (CONTINUED) 
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of personnel who reported having held a 9650 billet as a 
Lieutenant Colonel or Colonel (14) a change in the answer of 
one of these men (e.g., from "Sometimes" to "Always") would 
mean a frequency difference of more than 7% for any particular 
subject. Therefore, an arbitrary absolute difference ( | A | ) 
of 10% was chosen to indicate a "significant" difference in 
frequency of reported usage ("Sometimes" plus "Always") for 
each of the two groups. Thus, the service rank group that 
showed a value for j A | of 10% or more in reported usage fre- 
quency for any particular subject was deemed to have the greater 
reported usage frequency for that subject. The results obtained 
by this method are shown in Table XII. 

The second comparison method was a relative difference 
approach. Reported frequencies ("Sometimes" plus "Always") for 
Captains/Majors, k, and for Lieutenant Colonels/Colonels, Z , 
were used to compute relative frequencies, k/Z. The values 
of k/Z above 1.25 (arbitrary) were judged to mean a higher 
usage frequency for Captains and Majors and those below 0.75 
were judged as higher frequencies for Lieutenant Colonels and 
Colonels. The results obtained by this method are reflected 
in Table XIII. 

Subjects that were indicated as being used with greater 
frequency by each rank group by both methods are shown in 
Table XIV. Two conclusions that might be drawn from this 
analysis are that it appears that the more junior officers are 
using economic theory and techniques (as well as Computer 
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TABLE XII 



GREATER FREQUENCY OF USAGE BY OFFICER RANK 
(ABSOLUTE DIFFERENCE) 



Captains/Majors 




Lieutenant Colonels/Colonels 


Itechnique 


]Aj% 


Itechnique 


M% 


Forecasting 


29 


Capital Budgeting 


31 


Economics 


28 


Policy Analysis 


23 


Computer Programming 


27 


Risk Analysis 


17 


Econometric Models 


17 


Cost Effectiveness 


16 


Linear Algebra 


14 


Combat Models 


16 


Cost Models 


13 


Data Analysis 




Networks, Flows and 




(Nonpar ame trie ) 


15 


Graphs 


10 


Gaming Models 


15 






Probability Theory 


11 






Combat Analysis 


11 





TABLE XIII 




GREATER FREQUENCY OF 


USAGE BY OFFICER RANK 




(RELATIVE 


DIFFERENCE) 




Captains/Ma j ors 




Lieutenant Colonels/Colonels 


Itechnique 


k/Z 


Itechnique 


k/Z 


Forecasting 


3.071 


Capital Budgeting 


0.380 


Econometric Models 


2.214 


Integer and Mixed 




Economics 


2.000 


Progranming 


0.429 


Computer Programming 


1.750 


Policy Analysis 


0.540 


Heuristic Programming 


1.714 


Dynamic Progranming 


0.571 


Linear Algebra 


1.483 


Nonlinear Programming 


0.619 


Cost Models 


1.302 


Gaming Models 


0.700 






Risk Analysis 


0.702 






Combat Models 


0.719 



TABLE XIV 

GREATER FREQUENCY OF USAGE BY OFFICER RANK 


Captains/Ma j ors 


(BOTH METHODS) 

Lieutenant Colonels/Colonels 


Technique 


Itechnique 


Forecasting 


Capital Budgeting 


Economics 


Policy Analysis 


Econometric Models 


Risk Analysis 


Computer Progranming 


Gaming Models 


Linear Algebra 


Combat Models 



Cost Models 
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Programming and Linear Algebra) more often than their seniors 
are and that the senior officers are more involved with the 
budgeting of funds and the development of policy than are the 
junior officers (as one might expect) . 



K. BREADTH OF USAGE BY OFFICER RANK 

Having studied the differences between service rank groups 
in regard to usage frequency of individual OR techniques and 
application areas, we now look at whether there is a difference 
between the two officer groups based upon overall breadth of 
usage of the listed subjects. An index of breadth of usage was 
created by computing an average usage score for each individual 
in the following manner: a value of 1, 2 or 3 was assigned to 

each box marked "Always", "Sometimes" or "Never", respectively, 
in question 7 and these were summed over all techniques and 
application areas. This sum was then divided by 35, the total 
number of OR subjects listed. For example, an individual 
questionnaire with question 7 having 10 subjects marked as 
being used "Always", 15 subjects marked as being used "Some- 
times" and 10 subjects marked as being used "Never" would re- 

suit in an average usage score of i = 2.0 

for that particular individual. 

The complete list of all individual average scores was 
divided into two independent samples based upon service rank 
(Captains/Majors and Lieutenant Colonels/Colonels). A Kolmo- 
gorov-Smirnov (K-S) two-sample test was set up to test the 
null hypothesis that there is no difference between Captains/Majors 
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and Lieutenant Colonels/Colonels based upon breadth of usage 
of the OR techniques and application areas listed in question 
7 of the survey. Simply stated, a K-S test determines if the 
cumulative distributions of the two samples are sufficiently 
divergent to suggest that the samples come from different 
populations . 

The sizes of the samples in this case varied from relatively 
large (52 Captains and Majors) to relatively small (14 Lieu- 
tenant Colonels and Colonels) . Since adequate statistical 
tables for the K-S test statistic were not available for a 
mixture of large and small samples, two-tailed tests for both 
small and large samples were conducted resulting in acceptance 
of the null hypothesis. 

L. IMPORTANCE BY OFFICER RANK 

Examination of these same 35 subjects with regard to the 
perceived importance to the two service rank groups (Captains/ 
Majors and Lieutenant Colonels/Colonels) resulted in the fre- 
quency tables presented in Appendix D. Figure 4 exhibits com- 
parisons for each of the listed techniques and application 
areas for these two groups. The cross-hatched column over 
each subject indicates the percentage of Captains and Majors 
who reported an importance level of "1", "2" or "3" (average 
or more importance) for that subject and the clear enclosed 
columns denote the percentage of Lieutenant Colonels and 
Colonels who reported an importance level of "1", ”2" or "3" 
for each subject. 
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Figure 4. SERVICE RANK IMPORTANCE COMPARISON 
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Figure 4. (CONTINUED) 
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Figure 4. (CONTINUED) 



Absolute and relative difference methods similar to those 



used to determine frequency of usage differences were used to 
determine "significant" differences between the two officer 
rank groups regarding perceived importance of each OR tech- 
nique. For the absolute difference method, it was decided 
that the service rank group displaying a 10% or greater abso- 
lute difference, (|A|), in cumulative frequency of average-or- 
more-importance answers ("1" plus "2" plus "3") for any listed 
OR technique would be sufficient to indicate that that subject 
was regarded as more important by one of the officer ranks. 
Table XV reflects the results of this procedure. 



TABLE XV 

GREATER IMPORTANCE BY OFFICER RANK 
(ABSOLUTE DIFFERENCE) 

Cap tains /Majors Lieutenant Colonels/Colonels 



Technqiue 


|A|% 


Technique 


I a| ! 


Maintenance and Repair 


24 


Nonlinear Programming 


18 


Econometric Models 


21 


Dynamic Programming 


17 


Cost Estimation 


17 


Integer and Mixed 




Forecasting 


15 


Programming 


14 


Networks, Flows and 




Calculus 


10 


Graphs 


12 






Reliability 


12 






Economics 


12 






Equipment Replacement 


11 







For the relative difference approach, ratios of cumulative 
frequencies ("1" plus "2" plus "3") for Captains/Majors over 
Lieutenant Colonels/Colonels (k/£) were computed for each of 
the listed OR techniques. Those techniques having a k/H 
ratio of 1.25 or higher were considered to be of greater 
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importance to the Captains and Majors. Those techniques dis- 
playing a ratio of 0.75 or lower were deemed to be of greater 
importance to the Lieutenant Colonels and Colonels. The 
results obtained by using this method are displayed in Table 
XVI. 



TABLE XVI 

GREATER IMPORTANCE BY OFFICER RANK 
(RELATIVE DIFFERENCE) 



Captains/Majors Lieutenant Colonels/Colonels 



Technique 


k A 


Technique 


k/i 


Econometric Models 


2.000 


Dynamic Programming 


0.414 


Maintenance and Repair 


1.857 


Nonlinear Programming 


0.500 


Inventory Theory 


1.429 


Integer and Mixed 




Forecasting 


1.429 


Programming 


0.517 


Transportation 


1.429 






Cost Estimation 


1.340 






Input-Output Models 


1.321 






Equipment Replacement 


1.314 







Subjects which were included on both lists are shown in 
Table XVII. It appears from this table that economic tech- 
niques and equipment-related applications are perceived to be 
more important to the Captains and Majors while the advanced 



TABLE XVII 



GREATER IMPORTANCE BY OFFICER RANK 
(BOTH METHODS) 



Cap tajns/Ma j or s 



Lieutenant Colonels/Colonels 



Technique 



Technique 



Cost Estimation 
Econometric Models 
Forecasting 
Equipment Replacanent 
Maintenance and Repair 



Nonlinear Programming 
Dynamic Programming 
Integer and Mixed Programming 
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mathematical techniques are held in higher esteem by the more 
senior officers. 

M. BREADTH OF IMPORTANCE BY OFFICER RANK 

Next, the question of a difference between the two service 
rank groups based upon perceived importance to the individuals 
of the entire list of OR techniques (i.e., the "breadth of 
importance") was addressed. An index of the breadth of 
importance was created and , once again , a Kolmogorov-Smirnov 
two-sample test was used. An average importance score for 
each individual was computed by assigning a value of 1, 2, 3, 

4 or 5 to each box marked "1", "2", "3", "4" or "5" in question 
8, then summing the values over all techniques listed and 
dividing by 35 . The average importance scores were then 
divided into two groups based on the service rank of each 
individual and a two-tailed K-S test for two independent sam- 
ples was conducted resulting in acceptance of the null hypothe- 
sis. The implication of this test is that there is no differ- 
ence between Captains/Majors and Lieutenant Colonels/Colonels 
in regard to their perceived feelings of importance of all of 
the listed OR techniques and application areas as_ a whole. 

N. BREADTH OF USAGE BY FUNCTIONAL AREA 

Having found no differences in either overall breadth of 
usage or overall breadth of importance of the listed subjects 
based upon the seniority of the respondents, the association 
of breadth of usage and functional area of the respondents was 
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next inspected. Respondents' average usage scores, as deter- 
mined by the method described above, were partitioned into the 
following general function areas: "Manpower", "Logistics", 

"Operations and Plans", "Test and Evaluation", "Aviation", 
"Combination" and "Other". A Kruskal-Wallis one-way analysis 
of variance was then employed to test the null hypothesis: 
there is no difference between overall breadths of usage of the 
OR techniques for individuals employed in the above-listed general 
function areas. The Kruskal-Wallis one-way analysis of variance 
was chosen for use because it requires only that the variable 
measurement be (at least) ordinal and that the variable have 
an underlying continuous distribution. In addition, the Kruskal- 
Wallis test is more efficient than some other nonparametric 
tests of population differences because it uses more of the 
information in the observations by preserving the magnitude of 
the individual scores [Ref. 14:pp. 193-194]. Utilizing this 
test and correcting for the effect of ties in the observations 

[Ref. 14:p. 188], a test statistic of 13.52 was computed. This 

2 

statistic is approximately x distributed with 6 degrees of 
freedom and yields a P value of 0.035. Thus, the null hypothe- 
sis is rejected, meaning that there is a difference between 
individuals (when they are grouped according to the functional 
area of their billets) based upon their overall usage of all 
of the listed OR techniques. 

Close examination of the means of the average usage scores for 
each function area (displayed in Table XVIII) may help us under- 
stand a possible cause for this null hypothesis rejection. The 
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TABLE XVIII 



FUNCTION AREA MEAN USAGE SCORES 



Function Area 


Mean Usage Score 


Sample 


Combination 


2.222 


8 


Aviation 


2.405 


6 


Manpower 


2.439 


11 


Test and Evaluation 


2.472 


14 


Operations and Plans 


2.543 


7 


Logistics 


2.589 


8 


Other 


2.590 


12 



mean of the average usage scores for each function area may be 
interpreted to represent a measure of the overall breadth of 
usage of the listed subjects by individuals employed in each 
of these general areas. The closer a group's mean usage score 
is to 1.0, the higher the overall frequency of usage of OR 
techniques by the individuals in that functional area (recall 
that "Always" = 1, "Sometimes" = 2 and "Never" = 3). There 
is a notable difference in mean breadth of usage for the 
groups, especially in the case where individuals worked in a 
combination of functional areas. Intuitively, we might expect 
that individuals working in a combination of functional areas 
would be apt to use OR techniques more than those whose work 
might be channeled into one particular area. 

0. BREADTH OF IMPORTANCE BY FUNCTIONAL AREA 

The same method as described in the previous section was 
employed to test for differences of breadths of importance of 
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the subjects to individuals employed in the function areas 
listed above. A P value of 0.036 was obtained, thus, again, 
causing rejection of the null hypothesis of no difference. 
Looking at the mean importance scores for each function area 
(displayed in Table XIX) helps to explain this rejection. A 
low score implies a high overall perception of importance of 
the listed subjects while a high score implies a low overall 
feeling of importance (recall that Extremely Important = 1 and 
Not Important = 5) . Once again we see a high variance of mean 
scores, ranging from 2.636 for those individuals working in a 
combination of functional areas to 3.912 for those working in 
the field of Logistics, and again, intuitively, we might expect 
that those individuals whose work is not confined to a particu- 
lar area would be more apt to feel that, overall, OR techniques 
are more important than those individuals restricted to working 
in a specific field. 

TABLE XIX 

FUNCTION AREA MEAN IMPORTANCE SCORES 
Function Area Mean Importance Score Sample Size 



Combination 


2.636 


8 


Manpower 


3.301 


11 


Aviation 


3.357 


6 


Test and Evaluation 


3.400 


14 


Other 


3.500 


12 


Operations and Plans 


3.845 


7 


Logistics 


3.912 


8 
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P. BREADTH OF USAGE BY TYPE OF OR WORK DONE 

The same procedures as described above were used to test 
for differences between overall breadth of usage and overall 
breadth of importance based upon the following types of OR 
work that the respondents reported doing the most: 

Original OR work 

Reviewing OR work done outside their section 
Any combination of the types of OR work listed in question 
4 of the survey 
Any other type of OR work 
None . 

The average usage scores for all respondents were divided up 
into these five groups and a Kruskal-Wallis one-way analysis 
of variance test was conducted for the following null hypothe- 
sis: there is no difference in breadth of usage of OR techniques 

for individuals required to do the above-stated types of OR work 
the most. A test statistic of 12.36 was obtained (having cor- 
rected for tied observations) resulting in a P value of 0.015, 
causing a rejection of the null hypothesis. This means that 
there is a difference in breadth of usage of OR techniques be- 
tween individuals doing different types of OR work. It can be 
seen from Table XX that mean usage scores varied from 2.378 
(relatively high usage of the techniques) for the group doing 
original OR work to 2.743 (relatively low usage of the tech- 
niques) for the group doing no OR work. This would seem to 
intuitively support the statistical findings of the test. 
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TABLE XX 



MEAN USAGE SCORES BY TYPE OF OR WORK DONE 



Type of OR Work 


Mean Usage Score 


Sample 


Original 


2.378 


22 


Other 


ro 

• 


10 


Review 


2.513 


23 


Combination 


2.567 


6 


None 


2.743 


5 



It is interesting to note that, while five individuals 
reported in question 4 that they did not do any OR work, they 
went on in question 7 to report that they used at least some 
of the listed OR techniques either "Sometimes" or "Always", 

Q. BREADTH OF IMPORTANCE BY TYPE OF OR WORK DONE 

The Kruskal-Wallis test was also employed to test for no 
difference of breadth of importance of the listed subjects 
for the individuals required to do the above-listed types of 
OR work the most. A P value of 0.068 was obtained, causing 
null hypothesis acceptance at the 0.05 level. This means 
that, while individuals may use the OR techniques to varying 
degrees based upon the type of work they do, there is not a 
significant degree of disagreement between these individuals 
based upon the overall perceived importance of knowing how 
to use (and when not to use) these techniques. 
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R. ADDITIONAL OR TECHNIQUES 

Numerous OR techniques and general subject areas were 
mentioned by the respondents when asked what techniques they 
used in addition to those listed in the questions. These sub- 
jects can be separated into four general categories: 

Management and Communications 
Problem Solving Procedures 
System Development 
Computer Science 

Twenty-five people expressed an importance for knowledge 
in the areas of Management and Communications. (Management 
techniques, contract management, general decision analysis, 
bureaucratic realities, management information systems, public 
speaking, technical report writing, communications systems, 
salesmanship, information theory and data communications were 
a few of the subjects specifically mentioned.) Fifteen indi- 
viduals listed subjects that might be incorporated under the 
general heading of Problem Solving Procedures. These included 
problem definition, data collection, quality control, data 
processing fundamentals, measures of effectiveness, scaling 
techniques and common sense. Subjects possibly encompassed by 
the title System Development were mentioned by 12 respondents. 
These subjects included system design, system acquisition, the 
Planning, Programming and Budgeting System and systems analysis. 
Finally, 9 individuals felt that specialized knowledge in the 
field of computers was of great assistance in aiding them in 
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the performance of their duties. Specifically, they listed 
computer modeling, computer program design, computer applica- 
tions and computer system design as being important to them. 

S. TIME SPENT DOING OR WORK 

When asked what percentage of their work they actually spend 

doing OR work in comparison to their other duties, the personnel 

surveyed responded as shown in Table XXI. Overall, it appears 

that the great majority of those answering (78.9%) spend twenty- 

five to seventy-five percent of their time in the actual con- 

2 

duct of Operations Research. In addition, using a x test for 
two independent samples, the null hypothesis that there is no 
difference between Captains/Majors and Lieutenant Colonels/ 
Colonels based upon percentage of work time actually spent doing 
OR work was tested. The resultant P value of 0.471 indicates 
acceptance of the null hypothesis, indicating that there is not 
a significant difference between the amount of work time- spent 
by the more senior officers in performing OR-related tasks 



TABLE XXI 



PERCENT OF WORK TIME ACTUALLY SPENT DOING OR WORK 



Percent 


Captains/Majors 


Lieutenant Colonels/Colonels 


Total 


0 


8 (15.4) 


2 (14.3) 


10 (15.1) 


25 


12 (23.1) 


5 (35.7) 


17 (25.8) 


50 


12 (23.1) 


5 (35.7) 


17 (25.8) 


75 


16 (30.8) 


2 (14.3) 


18 (27.3) 


100 


4 (7.7) 


0 (0.0) 


4 (6.1) 


Totals 


52 (100) 


14 (100) 


66 (100) 


Ave. Time 
Spent 


54.3% 


37.5% 


45.8% 



64 



and the amount of work time spent by Captains and Majors in 
these types of endeavors. 

The percentage-of-work-time-spent data was also partitioned 

into groups based upon general function area as shown in Table 

2 

XXII. While it is true that a x test should not be performed 
on this partitioning due to the fact that 23% of the expected 
frequencies fall below 1.0 and all are below 5.0, the wide 
variance of average time spent by individuals employed in each 
function area (from 12.5% for those involved in Logistics to 
78.1% for those placing their jobs in the category of a combina- 
tion of function areas) suggest that there is a difference be- 
tween individuals employed in each of the listed function 



TABLE XXII 

FUNCTIONAL AREA VERSUS PERCENT OF WORK TIME SPENT DOING OR WORK 




0% 2 4 

25% 1 4 

50% 5 0 

75% 3 0 

100 % _0 0 

Total 11 8 
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o 

d 

a 
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o 

0 

1 

5 

2 
8 
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1 

5 

2 

4 

_0 

12 
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8 
£ 

10 

17 

17 

18 
_4 
66 



Average 

Time 45.5 12.5 25.0 58.9 33.3 78.1 43.75 

Spent 
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areas based upon the percentage of work time actually spent 
doing OR work. 

Of the 27 individuals who reported spending 25% or less 
of their work time performing explicit OR functions, 9 stated 
that the majority of their time was spent performing general 
management functions, 7 reported that studies review or policy 
review (that did not require explicit OR work) consumed most 
of their time, 4 reported administrative duties and 3 said 
working with computers kept them busy most of the time. 



T. ACCEPTANCE OF OR WORK BY SUPERIORS 

Table XXIII contains the results of the response when 

individuals were asked to relate the basic level of acceptance 

of OR work by their superiors. As can be seen, the majority 

reported that OR work is usually accepted while none of the 

2 

respondents said that OR work is never accepted. A x test 
was used to test the null hypothesis that there is no 



TABLE XXIII 

OR WORK ACCEPTANCE LEVEL 



Acceptance Level 
Accepted w/o Question 
Usually Accepted 
Rarely Accepted 
Never Accepter 
Totals 



Captains/Majors 
10 (20.4) 

35 (71.4) 

4 (8.2) 

0 ( 0 . 0 ) 

49 (100)* 



Lt. Colonels/Colonels 
4 (28.6) 

9 (64.3) 

1 (7.1) 

0 ( 0 . 0 ) 

14 (100) 



Total 
14 (22.2) 
44 (69.8) 
5 (7.9) 

0 ( 0 . 0 ) 
63 (100)* 



* 

Note: 3 individuals failed to answer the question. 
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difference between Captains/Majors and Lieutenant Colonels/ 
Colonels based upon the acceptance level of OR work of those 
for whom they work. (The category of "Never accepted" was 
dropped from the test since the number of observations from 
both samples for this category was zero.) A P value of 0.81 
was obtained, resulting in acceptance of the null hypothesis. 
Thus, it appears that the acceptance level of OR work as 
perceived by the respondents is independent of the service 
rank of those proffering it. 



* 
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IV. COMMENT ANALYSIS 



The analysis described in the previous section relates 
directly to the first eleven survey questions, all of which 
can be described as being of a quasi-objective nature. Ques- 
tion 12, an open-ended solicitation of comments, resulted 
in billet utilization recommendations which can only be 
characterized as subjective. For this reason, the results 
obtained from question 12 will be discussed in this separate 
chapter. 

One should bear in mind when reading this analysis that 
in no case did a majority of the respondents address any 
single subject. Rather, all topic discussions have been 
derived from the comments made by two to, at most, thirteen 
individuals. The author's intent in this chapter is to re- 
late to the reader not so much the general consensus of the 
entire sample population as it is to give the flavor and 
substance of the returned comments. In this manner it is 
possible to illuminate some of the underlying issues regard- 
ing the use of Operations Analysts and, hence, to make possi- 
ble recommendations for policy changes. 

The most enlightening notion that question 12 evoked was 
that Operations Research is really a way of thinking which 
should be required of all Marine officers. Many respondents 
found their OR backgrounds to be almost as valuable to them 
in their assignments as line officers as in their SEP billet 
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tours. They felt that the real value of their advanced 
education was in providing them with an analytical method 
of formulating problems, thinking about causes and then 
determining their solutions in a systematic manner. Pure 
OR work is rare, they found, but techniques, such as mathe- 
matical programming, provide a good point of origin for 
thinking about specific problems (even though "a heuristic/ 
seat-of-the-pants approach is what is actually required"). 

Many individuals also stated that there is an apparent 
lack of understanding on the part of many senior officers 
in managerial positions as to what Operations Research is 
and what analysts have been trained to do. This unawareness 
often leads to misuse of the analysts' technical skills or, 
worse, skepticism as to the validity of analytical work. 

Two officers reported that OR is often confused with Computer 
Science and, consequently, they spend much of their time 
writing and revising computer programs that are not related 
to analysis, while many respondents remarked upon the apparent 
suspicion among some senior officers in the Corps when opera- 
tions analysis is suggested as a viable method of problem 
solving. Classical Operations Research techniques, such as 
those listed in the questionnaire, are often considered to 
be esoteric by senior officers so, consequently, 9650 billet- 
holders must often solicit OR work and then "sell" their 
results . 

The source of these problems seems to be a rather deep- 
seated, historical one and was succinctly described by one 
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Lieutenant Colonel who wrote: 



OR analysts are viewed with mixed emotions — the 
manipulation of analysis by Pentagon officials 
starting with McNamara 'Whiz Kids' who set out 
to prove the bottom line .still persists in PA&E 
[Program Analysis and Evaluation] . [These offi- 
cials] get their marching orders from the politi- 
cal appointees in the administration. Military 
analysts at that level are used by the services 
to conduct counter studies or to discredit work 
done by the administration if it is counter to 
service-perceived goals. We [Operations Analysts] 
are viewed as necessary evils and are tainted by 
the misuse of OR. 

One suggested solution to this dilemma was to educate 
the officers filling supervisory positions over the analysts. 
In fact, it was felt by many respondents that common, basic 
OR education would lead not only to a greater degree of 
acceptance of the work of OR trained technicians but also 
to a more productive officer corps as a whole. 

In regard to actual utilization of the billets, almost 
twenty percent of the respondents suggested that the pro- 
ductivity of OR billets might be improved by centralizing 
most of them in a single Operations Analysis Group. Being 
spread throughout the Marine Corps, as they are now, many 
analysts have found that the low incidence of OR work brought 
to them often does not justify their billets . This appeared 
to be particularly true at the Division/Wing level where the 
fast pace of daily activity brought about by what is per- 
ceived to be excessive demands often results in quick prob- 
lem response at a superficial level of analysis. This is not 
conducive to the methodical and measured thought process often 
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required for thorough and accurate analytical work. Conse- 
quently, analysts at the lower command levels often either 
become embedded in administrative oblivion and structure 
politics or they are simply converted into efficiency experts 
assigned the task of acting as command hatchet-men, seeking 
out waste and misuse of resources. 

Recommendations for correcting this deficiency were either 
to disestablish OR billets below the FMF level or, better, 
to centralize them within a single department. A central sec- 
tion, perhaps under the direction of a Deputy Chief of Staff 
for Operations Analysis, could accept viable projects submitted 
by other departments or commands throughout the Corps. Teams 
of one to six analysts (depending upon the size and scope of 
the problem) with appropriate backgrounds could be assigned 
to each project on an as-needed, as-available basis. This 
reorganization would, in the opinion of the respondents lead 
to better long-range study planning and a less hectic daily 
press for results. 

This recommendation goes counter to the findings of Thomas 
and DaCosta [Ref. 9], In their survey of private industry 
they found that the most recent trend in the private sector 
has been away from centralized Operations Research departments. 
Their survey disclosed that "many respondents felt that by 
spreading Management Science [Operations Research] throughout 
the firm, practitioners would be better acquainted with the 
problems they had to solve and, therefore, would be more 
efficient" [Ref. 9, p. 103]. 
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A number of individuals addressed the subject of OR work 
contracted out by the Marine Corps to civilian organizations. 
Many of the respondents expressed concern over the Marine Corps' 
apparent eagerness to issue contracts to outside sources who 
often charge exorbitant fees but who sometimes use incorrect 
OR techniques and/or produce questionable results. One indi- 
vidual, working in the area of Test and Evaluation, felt that 
"the Marine Corps is paying too much to 'Beltway Bandits' 
[civilian contractors] and their ilk for studies which are 
nothing more than mumbo jumbo". Many respondents felt that, 
if assigned to these tasks, they could do most of the techni- 
cal work themselves thus producing better results at a fraction 
of the costs. 

The general idea expressed was that, in order for Operations 
Analysts to be used most effectively, they should be "doers" 
rather than mere contract supervisors. If resource limita- 
tions prevent this, however (e.g., more work than can be done 
by available Marine analysts) then the analysts should at 
least be used to review the enti re process of contracted 
studies and not just the finished product. Marine analysts 
should be included in the original contract negotiations with 
civilian agencies to ensure that the Marine Corp’s is not led 
into allowing meaningless or unnecessary tests to be performed. 
Operations Analysts should monitor the contracted work as it 
progresses and should, certainly, inspect the final results. 
According to the respondents, this monitoring and review work 
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must be a comprehensive quality control function. If the role 
filled by the 9650 is merely that of "riding herd" over previ- 
ously contracted studies, then it was felt that an individual 
with a non-technical background such as a Systems Analyst 
(MOS 9652) could much more efficiently fill the billet. 

The greatest number of comments received addressing any 
single subject (13) were those that recommended a reevaluation 
of the requirements for 9650 billets. While an annual review 
of all SEP billet requirements is conducted by HQMC , the 
respondents, generally, did not believe it to be comprehen- 
sive enough. Two main reasons were perceived by the respon- 
dents as being responsible for the unnecessary establishment 
of billets: (1) The billets were originally requested and 

approved by individuals who really didn't understand Operations 
Research and its applications and (2) the desire by some senior 
officials to add intelligent (but possibly unnecessary) offi- 
cers to their staffs. One of the results of this billet 
inflation is that, very often, a promising, career-oriented 
officer is assigned to a billet that requires him to use very 
little of his special training and expertise. In fact, so 
much time appears to be spent by analysts working on special 
but non-analysis related staff projects that some individuals 
expressed doubt as to the need for any OR billets. 

While there are, of course, billets that do require and 
utilize a working knowledge of OR techniques, it was rather 
obvious from the comments received that a truly comprehensive 
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review of all 9650 billet requirements would probably lead 
to the realization that some billets are either misused or 
not essential. Once identified, these billets could be dis- 
established or modified (e.g., replace a 9650 with a 9652) 
or the cause of misutilization could be corrected. The real 
problem, however, as viewed by those who have held the billets, 
would lie in the identification process. This would require 
a frank and honest review of the requirements for all 9650 
billets and, while it was generally felt that the analyst 
holding an "unessential" billet would probably acknowledge 
such, there was little faith that his command would make the 
same acknowledgment. In fact, several individuals expressed 
concern that there might be retribtuion in the form of poor 
fitness reports if an organization lost any billets due to 
the honest evaluation of an incumbent. One respondent sum- 
marized the feelings expressed by several by writing. 

What the Marine Corps needs is a truly covert, 
non-attributable billet validation procedure. No 
bureaucratic organization is going to give up a 
body voluntarily unless it can get something in 
trade of equal or greater benefit, i.e., the organi- 
zation will always validate the billet. If I am 
scheduled to leave and my boss is told by HQMC 
(He starts calling. . . long before a person is 

actually scheduled to leave, demanding . . . replace- 

ment.) that they have decided not to fill this 
billet anymore, then his automatic inference is 
that I 'dropped the dime' and he gives appropriate 
award on my parting fitness report. Hence, what 
is needed is covert elicitation of billet vali- 
dation from incumbents and protection in those 
cases where billets cannot be validated and will 
be eliminated (e.g., stall and stonewall on 
replacement) . 

If such a system of annual validation could be established, 
several individuals felt that as many as one-third of the 
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present number of 9650 billets might be made available for 
more productive use. (Indeed, it was seen above that almost 
41% of the respondents stated that they spent 25% or less of 
their work time actually performing OR work.) 

Two individuals did state that the Marine Corps needs more 
9650 billets. They declared that there are many people (who 
are unaware of the value of applying OR techniques) attempting 
to solve problems which could be handled much more efficiently 
using analytical methods. Headquarters, Marine Corps apparently 
has a virtual gold mine of OR solvable problems just waiting 
to be recognized by someone versed in the field. What is 
needed, according to these individuals, is enough trained 
analysts to ferret these problems out and solve them. 

Finally, two individuals recommended that prospective OR 
graduates be paired with their initial billet assignments as 
early as practical (at least two quarters prior to graduation 
and, ideally, a year prior) . The current SEP monitor at HQMC 
realizes the importance to students of early billet notifica- 
tion and is attempting to do this now. Certainly, this does 
require planning and foresight on his part but unforeseen 
difficulties do arise and changes in prospective assignments 
must sometimes be made. Still, early notification does allow 
selection of elective courses oriented toward the billet to 
be filled and permits proper preparation on the part of the 
students. These students should bear in mind, however, that 
subsequent 9650 tours might be related to entirely different 
function areas than their initial billet assignments; hence, 
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they should not design elective programs restricted solely 
to one field but should try to maintain at least a minimum 
balance of OR subjects. 

Some of the most interesting results of the survey were 
those obtained from the answers to question 12 as described 
in this chapter. Although usually no more than four or five 
of the 72 responding individuals addressed any one particular 
subject, many of the remarks appeared to be well thought-out 
and reflected the advantage of being based upon actual experi- 
ence. It should be reemphasized, however, that, simply be- 
cause one, two or even six individuals voiced an opinion, it 
does not necessarily mean that their comments are reflective 
of the entire population. Even with this caveat of subjectivity, 
however, it is possible to draw attention to the issues that 
have been raised and to make subsequent recommendations re- 
garding them. This is done in the next chapter subsequent 
to a short comparison of the results of this study with those 
of studies done previously. 
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V. COMPARISONS, CONCLUSIONS AND RECOMMENDATIONS 



A. COMPARISONS WITH PREVIOUS STUDIES 

Since the format and conduct of this study did not exactly 
match those of investigations done previously, including Obert's 
survey of Army OR technique usage, the results cannot be 
collated in a parallel manner. However, some comparisons 
can be made . 

Data analysis and statistical methods were the most fre- 
quently used OR techniques as in all the studies reviewed 
in Chapter I. Linear Programming was reported to be used by 
only about a third of the Marine Corps respondents, matching 
the results found in Obert's study of Army OR/SA engineers 
but falling quite a bit short of the apparent wide-spread usage 
reported by private firms (where LP-amenable problems such as 
resource allocation are possibly more numerous and better de- 
fined) . Inventory Theory and the more advanced mathematical 
programming subjects were generally listed higher in frequency 
and importance orderings for corporate industry than they were 
in this study while Risk Analysis was found to be more widely 
used by Marine analysts than by private firms as reported by 
Thomas and DaCosta. Once again, this is probably a reflection 
of the natures of the two organizations. Private firms are 
more concerned with the resource allocation and sales problems 
while military operations characteristically require a high 
level of risk-taking. 
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Some of the most interesting and, perhaps because of the 
nature of employment of the persons surveyed, some of the 
most appropriate comparisons can be made between this study 
and the one conducted by Obert. The author found that a 
smaller percentage of Marines than Army officers: (1) Con- 

ducted original OR work, correlated OR work or reviewed OR 
work (74% to Obert's 87%). (2) Worked with established com- 
puter programs (66.7% to Obert's 80.3%). (3) Reported spend- 

ing 50% or less of their work time doing OR work (66.7% to 
Obert's 73.2%). Also, unlike Obert's study, this investiga- 
tion found that there is not a significant difference between 
the amount of work time spent by Captains/Majors and the 
amount of work time spent by Lieutenant Colonels/Colonels in 
the performance of OR-related tasks. Operations Research 
techniques which were found to be placed in nearly equal 
positions in orderings of frequency of use by Marine analysts 
and Army OR/SA engineers included Parametric Statistics, Pro- 
bability Theory, Networks, Human Factors, Linear Algebra, 
Linear Programming, Non-Linear Programming and Dynamic Pro- 
gramming. Techniques which reflected a more widespread use 
* by Army officers than by Marines included Combat Models and 
Decision and Risk Analysis, while the Marines reported greater 
usage of Experimental Design, Nonparametric Statistics, Cost 
Estimation, Calculus, Economics, Manpower Models and Input/ 
Output Models. Barring these differences, however, the remain 
der of the results of this work were relatively similar to 
those of Obert's study. 
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B. CONCLUSIONS 



This section includes the main conclusions drawn from the 
analysis of the data and comments obtained from the returned 
questionnaires. 

1. Data Analysis 

Of all the OR techniques listed in the questionnaire, 
the more advanced mathematical programming methods such as 
Nonlinear, Dynamic, Heuristic, Integer and Mixed Programming 
are used the least and believed to be the least important 
by the surveyed analysts in carrying out their daily duties 
as 9650‘s. Parametric Data Analysis, Probability Theory, Cost 
Effectiveness and Networks are the subjects used the most and 
believed to be the most important. 

While it was found that there are differences in usage 
of individual techniques according to service rank groups 
(Captains and Majors use economic theory and related techniques 
such as Econometric Models, Cost Models and Forecasting as well 
as Linear Algebra and Computer Programming more often than do 
Lieutenant Colonels and Colonels while the latter use Risk 
Analysis, Gaming Models, Combat Models, Capital Budgeting and 
Policy Analysis more often than do the former) , on an overall 
basis there is no statistical difference between the two groups 
based upon the breadth of usage of all of the OR techniques 
and application areas. What, then, might be possible causes 
for differences in overall breadth of usage? 

It was found that there are statistically significant 
differences between breadths of usage of the techniques based 
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upon the general function areas related to the 9650 billets 
and based upon the type of OR work that the analysts were 
required to do the most. As might be expected, individuals 
working in a combination of functional areas and those per- 
forming original OR work were found to use a wider variety of 
OR techniques than those analysts employed in separate func- 
tional areas or those performing various other types or com- 
binations of OR work. 

Regarding perceived importance of the techniques and 
application areas listed in the questionnaire, individual 
differences also exist based upon the two service rank groups. 
Economic models (Econometric Models, Cost Estimation and Fore- 
casting) as well as Equipment Replacement and Maintenance and 
Repair are perceived to be more important by Captains/Majors 
than by Lieutenant Colonels/Colonels while, on the other hand, 
most of the advanced mathematical programming methods (Non- 
linear, Dynamic, Integer and Mixed Programming) are perceived 
to be more important by the Lieutenant Colonels and Colonels. 

Once again, however, no difference was found to exist 
between the two service rank groups based upon overall breadth 
of importance of all the techniques, nor was a difference found 
based upon the different types of OR work required to be per- 
formed by the individuals. (This might imply that the respon- 
dents placed a relative equality of importance on the techniques 
whether they used them or not.) The only possible causes for 
differences in overall breadth of perceived importance of the 
listed techniques was that attributed to variation in the general 
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function areas related to the 9650 billets. Those individuals 



working in the areas of Logistics and Operations and Plans 
assigned the lowest breadth of importance value to the tech- 
niques while those working in a combination of areas assigned 
the highest breadth of importance value. 

Two other determinations were made regarding the usage 
and importance of techniques. First, there is a strong corre- 
lation between reported utilization of and perceived importance 
of the 35 OR techniques listed in the questionnaire (which is 
rather intuitive — the more a technique is used, the more likely 
it is that the user will find it to be important) . Secondly, 
although most Marine analysts are not required to write their 
own computer programs, those who do write their own use Fortran 
as their most frequent programming language. (It was noted 
that most analysts do, however, interact with established or 
"canned" programs) . 

Analysis was also conducted to determine reasons for 
possible differences in the type of OR work required to be 
done by the 9650's, the percentage of work time Marine analysts 
spend doing OR work and the level of acceptance of OR work 
by those for whom the respondents worked. No differences in 
any of these three areas were found based upon the two service 
rank groupings (Captain/Major and Lieutenant Colonel/Colonel) . 
However, the type of OR work performed was found to vary 
according to billet-related functional areas. (It is inter- 
esting to note that OR work is, at least, usually accepted by 
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most superiors and in no case was it found to be consistently 
not accepted . ) 

2 . Comment Analysis 

It is rather more difficult to draw hard and fast 
conclusions from the comment analysis section than from the 
data analysis section of this study. The subjective nature 
of the comments, as well as the fact that none of the topics 
discussed were mentioned by a majority of the respondents 
direct us to accept the fact that, rather than producing con- 
clusions, the comments really simply raise issues that require 
further study. 

The considerable response that was obtained when the 
analysts were asked to make recommendations for improving the 
utilization of 9650's appears to reflect at least some measure 
of job dissatisfaction on the part of the respondents. The 
main cause of this disappointment is probably the current mis- 
use of some Marine Corps analysis resources. A strategy for 
alleviating this misuse can be developed by addressing the 
three major issues brought out by the comments. These are: 

(1) The need for education of 9650 billet supervisors. (2) The 
need for education of Marine Operations Analysts. (3) The 
need for reevaluation and possible reorganization of current 
9650 billets. 

a. Educating Supervisors 

It is clear from the comments that there is some 
organizational resistance to the initiation and implementation 
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of OR work. This was judged to be due mainly to the misunder- 
standing on the part of some senior Marine officers as to what 
OR really is and what its value can be in the decision-making 
process. While it is understandable that a number of super- 
visors will be unversed in the more mundane esoterica of Opera- 
tions Research, it should certainly be desirable to educate 
these officers so that the goal of proper utilization of analyst 
resources might be attained. Just as an infantry officer should 
be knowledgeable in the proper deployment and employment of 
his troops, so should a supervisor of Operations Analysts be 
capable of intelligently assigning and employing his personnel 
resources . 

A more difficult problem to counter is that of en- 
grained resistance to change and/or new ideas due to limited 
vision or funneled perception. Some individuals are simply 
opposed to innovative but obscure approaches to problems simply 
because their past experiences have included only simple, "head- 
on" approaches. While it may certainly be argued that in many 
cases the direct approach is the best course of action, super- 
visors should at least be open to suggestions concerning novel 
methods of reaching problem solutions. 

b. Educating Operations Analysts 

One factor contributing to 9650 job dissatisfaction 
is the frustration felt by the analyst when he is unexpectedly 
confronted by the realities of analytical work in the "real 
world". This is especially true in the cases of young analysts 
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who have recently finished their graduate work and are zealously 
approaching their first assignments confident that they are 
capable of performing creditable analytical work. What they 
are often surprised to find is that analysis in real life is 
not conducted in the same pristine and aseptic environment 
that they often enjoyed when approaching problems in the class- 
room. In fact, they may very quickly learn that their particu- 
lar billet assignment requires them to do very little, if any, 
real analytical work or that they might spend several weeks 
on a project only to have the results ignored by decision- 
makers or, even worse, that they might see good analytical 
work disregarded for nontechnical reasons. The prospective 
9650 should be made aware of the true nature of the environ- 
ment that he is about to enter prior to his arrival at his 
initial assignment. He must understand that he may encounter 
institutional resistance to OR work and that, in the real world 
of give-and-take, compromises must sometimes be made for’ politi- 
cal reasons. Additionally, there will be times when the Opera- 
tions Analyst must act as a technological change agent, soli- 
citing problems that can be solved by OR techniques and then 
actively selling his results to the decision-makers. By edu- 
cating the analyst in regard to occupational realities it might 
be possible to mitigate the frustrations he might feel because 
of unfulfilled expectations. 

c. Reevaluation and Reorganization of Billets 

There appear to be cases where Marine Operations 
Analysts are being improperly utilized. Disenchantment was 
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expressed by the 9650 's because of low OR work incidence, 
limitations of available analytical resources (both personnel 
and computational hardware) and simple misunderstandings con- 
cerning the capabilities and limitations of analysis. They 
felt that spreading OR billets throughout the Marine Corps 
can, in some instances, lead to inefficiency and misuse of 
resources and that, at times, OR- related projects are unneces- 
sarily contracted out by the Marine Corps to civilian agencies 
while the "in-house" analysts who could do the work themselves 
are assigned as Project Officers and relegated to a role of 
simple supervision and review. The opinion of many of the 
respondents was that 9650 billets are overdue for a hard, 
honest reevaluation of their technical skill requirements and 
organizational locations. 

C . RECOMMENDATIONS 

Based upon the conclusions drawn from the statistical analy- 
sis of the data and the issues raised by the comments submitted 
by the respondents, the author makes the following recommendations. 

Because of the relatively large number of comments received 
addressing the apparent need for a thorough reevaluation of 
9650 billet requirements, it is recommended that, during the 
next several required annual reviews of billet educational 
requisites, the Operations Research discipline sponsor (Deputy 
Chief of Staff for Research, Development and Studies) elicit 
candid comments from MOS 9650 billet incumbents regarding their 
perceptions of proper billet utilization. These comments should 
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then be incorporated into the normal billet review procedure 
to enhance the picture obtained by the discipline sponsor 
concerning the true requirements for the billets. 

Additionally, and perhaps concurrently with the review of 
educational requirements, a detailed study should be conducted 
to determine if Marine Corps OR billets should be organized 
on a more centralized basis. Where the requirement for sepa- 
rately assigned OR sections or individuals cannot be justi- 
fied, it might prove to be more efficient to group them into 
a single section, perhaps as an Operations Analysis Group under 
the control of the Deputy Chief of Staff for Research, Develop- 
ment and Studies. Under this organization, the conduct of 
studies might then be directed more effectively. In addition, 
an overall plan for reviewing original negotiations between 
the Marine Corps and civilian contractors, monitoring the 
progress of the contracted studies and inspection of the final 
results could be developed and employed by this group. 

A course of instruction regarding the management of OR 
resources should be offered to and taken by all individuals 
(including 9650's) who are or will be responsible for super- 
vising Marine Operations Analysts. This course could include 
such subjects as the Potential and Limitations of Operations 
Analysis, Management Techniques, Contract Management and 
Bureaucratic Realities. It could be taught at the Marine Corps 
Command and Staff College, should be at least one to two weeks 
long and could be taken by officers immediately prior to 
their arrival for duty in a supervisory position. 
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A short course in basic Operations Research could also 
be set up by the Marine Corps Extension School located at 
MCDEC, Quantico, Virginia. It should embrace, at a minimum, 
the very basics of OR to include foundational Probability 
Theory, simple Data Analysis and Statistical Inference, the 
theory behind Linear Programming, PERT and CPM, Design of 
Experiments and Technical Report Writing. This course could 
be made available to all Marines but should be specifically 
recommended to those officers who are filling 9650 billets 
but who do not possess the 9650 secondary MOS . 

For those Marine officers attending the Naval Postgraduate 
School as a prerequisite to obtaining a 9650 secondary specialty, 
the course of instruction contained in Appendix E is submitted 
as a means of preparing themselves for the requirements that 
they will find subsequent to graduation. This program is cer- 
tainly not all-inclusive nor does it take into consideration 
all of the driving and limiting factors that the administration 
of the Naval Postgraduate School must deal with when developing 
curricula. However, this program does contain most of the 
courses necessary to give the student at least a minimal 
knowledge in each of the areas found by this study to be 
commonly used and of importance. In addition, it is recommended 
that the Operations Research Department Chairman of the Naval 
Postgraduate School take into consideration either the estab- 
lishment of a new course dealing specifically with such topics 
as Advocacy Analysis and Bureaucratic Realities of Analysis 
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or at least consider increasing the emphasis placed upon such 
subjects in existing NPS courses. 

Finally, it is recommended that the Special Education Pro- 
gram monitor continue to place emphasis upon the early iden- 
tification of 9650 billet vacancies and the pairing of these 
openings with prospective analysts while they are still attending 
school. These pairings should be accomplished (understanding 
that last minute changes may become necessary) at least six 
months to a year prior to the graduation of the students. 
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APPENDIX A 



USMC OPERATIONS RESEARCH SURVEY 



If you are curxently serving in a MOS 9650 billet, please base your 
answers to all questions on present billet experiences. If you are not 
currently serving in a MOS 9650 billet but have done so in the past, please 
base your answers to all questions on experiences from your last 9650 
billet assignment. If you have never served in a MOS 9650 billet, please 
state so now and do not answer any further questions. Otherwise, please 
continue. 



i. a. What is the title of your 9650 billet? 



b. With what command is this billet located? 



2. What is (was) your rank while serving in this 9650 billet? 

3. Which of the following general function areas is this 9650 billet related 
to? (Circle one) 

a. Manpower e. Test and Evaluation 

b. Logistics f. Aviation 

c. Finance g. Instruction 

d. Operations and Plans h. Other: What area? 



4. What type of Operations Research (OR) work does this 9650 billet require 
you to do the most? (Circle one) 

a. ’ Original OR work 

b. Correlate the work of other 
researchers in your section 

c. Review OR work done outside 
your section 

5. Does this billet assignment require you to do your own computer programming? 



d. Instructor 

e. Other OR work 

f . Do not do any OR work 



a. If so, what computer language(s) do you use? 



b. Why do you use this particular language? 



6. Do you interact with any established computer programs (i.e. , preprogrammed, 
"canned", manufacturer-provided software)? 



89 



How often do you use the following OR techniques and application areas? 
(Place an X in the appropriate box) 



1) 

2 ) 

i) 

6 ) 

7) 

3) 

$ 

SJ 

13) 

14 ) 

15) 



b. 



1 ) 

2 ) 

3, 

4 

5 . 

6, 

7 ) 

3) 



1 ) 

2 ) 

3) 

4' 

6 

7 

3 

9 

10 

11 

12 



leral OR Techniques 


NEVER 


SOMETIMES 


ALWAYS 


Linear Programming 








Nonlinear Programming 








Dynamic Programming 








Integer and Mixed Programming 








Heuristic Programming 








Bayesian Decision Analysis 








Networks, Flows and Graphs (PSRT/CPM) 








Risk Analysis 








Cost Estimation 








Cost Effectiveness 








Reliability 








Inventory Theory 








Data Analysis and Statistical Inference 
(parametric} 








Data Analysis and Statistical Inference 
(Nonparametric) 








Probability Theory 








Modeling Techniques 








Queueing Theory and Stochastic Models 








Gaming Models 








Combat Models 








Manpower Models 








Econometric Models 








Cost Models 








Input -Output Models 








Forecasting 








ler Related Subjects 








Calculus 








Linear Algebra 








Economics 








Capital Budgeting 








Policy Analysis 








Combat Analysis 








Equipment Replacement 








Maintenance and Repair 








Transportati on 








Human Factors 








Design, Conduct and Evaluation of Experiments 








Computer Programming 
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8. What is the relative importance of having a working knowledge of these 
OR techniques and application areas in discharging the daily duties of 
your billet? (On a scale of 1=' Extremely Important 1 to 5= ’Not Important 
At AH'), Place an X in the appropriate box for each technique or subject, 

Extremely Not 

Important Important 



a. General OR Techniques 


1 


2 


3 


4 


5 


i; 


Linear Programming 












2) 


Nonlinear Programming 












3 ) 


Dynamic Programming 












4) 


Integer and Mixed Programming 












5) 


Heuristic Programming 












6 


Bayesian Decision Analysis 












7) 


Networks, Flows and Graphs (PERT/CPM) 












8 j 


Risk Analysis 












9 


Cost Estimation 












10 


Cost Effectiveness 












11 


\ Reliability 












12 


) Inventory Theory 












13 J 


1 Data Analysis and Statistical Inference 
(Parametric) 












14) Data Analysis and Statistical Inference 
( Nonparame tri c ) 












15) Probability Theory 












b. OR Modeling Techniques 












1] 


) Queueing Theory and Stochastic Models 












2) 


► Gaming Models 










_ 


3 


) Combat Models 












4 


1 Manpower Models 












5 


1 Econometric Models 












6 


1 Cost Models 












7 


) Input-Output Models 












8 


\ Forecasting 












c. Other Related Subjects 












1 


1 Calculus 












2’ 


> Linear Algebra 












3] 


) Economics 












4 


> Capital Budgeting 












3, 


) Policy Analysis 












5 ; 


) Combat Analysis 












7 { 


) Equipment Replacement 












a) 


) Maintenance and Repair 












9 ; 


) Transportation 












10 


) Human Factors 












11 


) Design, Conduct and Evaluation of Experiments 












12 ; 


) Computer Programming 
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9. Other than those listed above, what OR techniques and/or application 
areas do you feel you should have a working knowledge of in order to 
efficiently do your job? (Please list the top five in order of 
importance, ) 



10. a. Approximately what percentage of your work time is actually spent 

doing OR work in comparison to your other duties? (Circle one) 

1 ) 0 * *0 75 % 

2 ) 25 % 5 ) 100 % 

3 ) 30 % • 

b. If your answer to question 10. a. was l) or 2), in what area is most 
of your work time spent? 

11. Gener ally | what is the acceptance level of those whom you work for 
towards OR work? (Circle one) 

a. OR work is accepted without question 

b. OR work is usually accepted 

c. OR work is rarely accepted 

d. OR work is never accepted 

12. In your opinion, how could OR billets be better utilized? (Please use 
the reverse side if more space for response is required. ) 



13 . Do you wish to receive a copy of the results of this survey? 
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APPENDIX B 



OR TECHNIQUE USAGE AND IMPORTANCE FREQUENCIES 



USAGE Raw Score Percentage 




a. GENERAL OR TECHNIQUES 



1) 


Linear Programming 


43 


23 


0 


65 


35 


0 


2) 


Nonlinear Programming 


56 


10 


0 


85 


15 


0 


3) 


Dynamic Progr arming 


57 


8 


1 


86 


12 


2 


4) 


Integer and Mixed Programming 


61 


5 


0 


92 


8 


0 


5) 


Heuristic Programming 


59 


6 


1 


89 


9 


2 


6) 


Bayesian Decision Analysis 


48 


18 


0 


73 


27 


0 


7) 


Networks Flows and Graphs 


14 


49 


3 


21 


74 


5 


8) 


Risk Analysis 


37 


26 


3 


56 


40 


5 


9) 


Cost Estimation 


18 


43 


5 


27 


65 


8 


10) 


Cost Effectiveness 


13 


43 


10 


20 


65 


15 


11) 


Reliability 


25 


33 


8 


38 


50 


12 


12) 


Inventory Theory 


52 


12 


2 


79 


18 


3 


13) 


Data Analysis and Statistical 
Inference (Parametric) 


13 


38 


15 


20 


58 


23 


14) 


Data Analysis and Statistical 
Inference (Nonparametric) 


22 


31 


13 


33 


47 


20 


15) 


Probability 


15 


37 


14 


23 


56 


21 



b. OR MODELING TECHNIQUES 



1) 


Queueing Theory & Stoch. Models 


40 


25 


1 


61 


38 


2 


2) 


Gaming Models 


41 


22 


3 


62 


33 


5 


3) 


Combat Models 


37 


25 


4 


56 


38 


6 


4) 


Manpower Models 


38 


22 


6 


58 


33 


9 


5) 


Econometric Models 


48 


15 


3 


73 


23 


5 


6) 


Cost Models 


31 


31 


4 


47 


47 


6 


7) 


Input-Output Models 


42 


23 


1 


64 


35 


2 


8) 


Forecasting 


42 


21 


3 


64 


32 


5 



c. OTHER RELATED SUBJECTS 



1) 


Calculus 


32 


30 


4 


48 


45 


6 


2) 


Linear Algebra 


40 


23 


3 


61 


35 


5 


3) 


Economics 


33 


26 


7 


50 


40 


11 


4) 


Capital Budgeting 


49 


14 


3 


74 


21 


5 


5) 


Policy Analysis 


45 


15 


6 


68 


23 


9 


6) 


Combat Analysis 


34 


26 


6 


52 


40 


9 


7) 


Equipment Replacement 


32 


31 


3 


48 


47 


5 


8) 


Maintenance and Repair 


32 


30 


4 


48 


45 


6 


9) 


Transportation 


38 


28 


0 


58 


42 


0 


10) 


Human Factors 


31 


29 


6 


47 


44 


9 


11) 


Design, Cond. & Eval. of Exps. 


20 


35 


11 


30 


53 


17 


12) 


Computer Programing 


28 


26 


12 


42 


40 


18 
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IMPORTANCE 




TECHNIQUE 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


a. 


GENERAL OR TECHNIQUES 
























1) 


Linear Programning 


4 


7 


18 


13 


24 


6 


11 


27 


20 


36 




2) 


Nonlinear Programning 0 


4 


10 


18 


34 


0 


6 


15 


27 


52 




3) 


Dynamic Programning 


0 


1 


9 


24 


32 


0 


2 


14 


36 


48 




4) 


Integer & Mixed 
Programning 


0 


1 


11 


15 


39 


0 


2 


17 


23 


59 




5) 


Heuristic Programning 2 


1 


8 


13 


42 


3 


2 


12 


20 


64 




6) 


Bayes. Dec. Analy. 


1 


7 


14 


14 


30 


2 


11 


21 


21 


45 




7) 


Nstworks, Flows 
and Graphs 


5 


22 


21 


11 


7 


8 


33 


32 


17 


11 




8) 


Risk Analysis 


5 


15 


16 


11 


19 


8 


23 


24 


17 


29 




9) 


Cost Estimation 


12 


13 


17 


12 


12 


18 


20 


26 


18 


18 




10) 


Cost Effectiveness 


15 


16 


18 


10 


7 


23 


24 


27 


15 


11 




11) 


Reliability 


10 


17 


17 


12 


10 


15 


26 


26 


18 


15 




12) 


Inventory Theory 


2 


7 


10 


22 


25 


3 


11 


15 


33 


38 




13) 


Data Analysis and 
St at. Inf. (Param. ) 


22 


17 


12 


7 


8 


33 


26 


18 


11 


12 




14) 


Data Analy. & Stat. 
Inf. (NOnparametric) 


21 


15 


9 


11 


10 


32 


23 


14 


17 


15 




15) 


Probability Theory 


22 


19 


9 


6 


10 


33 


29 


14 


9 


15 


b. 


OR MODELING TECHNIQUES 
























1) 


Queueing Theory & 
Stochastic Models 


2 


12 


15 


18 


19 


3 


18 


23 


27 


29 




2) 


Gaining Models 


5 


16 


9 


15 


21 


8 


24 


14 


'23 


32 




3) 


Conbat Models 


9 


12 


6 


12 


27 


14 


18 


9 


18 


41 




4) 


Manpower Models 


12 


5 


13 


18 


18 


18 


8 


20 


27 


27 




5) 


Econometric Models 


2 


11 


12 


19 


22 


3 


17 


18 


29 


33 




6) 


Cost Models 


8 


19 


13 


14 


12 


12 


29 


20 


21 


18 




7) 


1-0 Models 


5 


11 


7 


16 


27 


8 


17 


11 


24 


41 




8) 


Forecasting 


7 


13 


11 


12 


23 


11 


20 


17 


18 


35 


c. 


OTHER RELATED SUBJECTS 
























1) 


Calculus 


8 


17 


13 


11 


17 


12 


11 


20 


17 


26 




2) 


Linear Algebra 


4 


10 


24 


9 


19 


6 


15 


36 


14 


29 




3) 


Economics 


11 


15 


13 


14 


13 


17 


23 


20 


21 


20 




4) 


Capital Budgeting 


2 


9 


14 


15 


26 


3 


14 


21 


23 


40 




5) 


Policy Analysis 


6 


11 


14 


16 


19 


9 


17 


21 


24 


29 




6) 


Combat Analysis 


9 


11 


13 


8 


25 


14 


17 


20 


12 


38 




7) 


Equipment Repl. 


2 


16 


11 


16 


21 


3 


24 


17 


24 


32 




8) 


Maintenance & Repair 


4 


12 


15 


16 


19 


6 


18 


23 


24 


29 




9) 


Transportation 


1 


13 


11 


21 


20 


2 


20 


17 


32 


30 




10) 


Human Factors 


11 


13 


13 


11 


18 


17 


20 


20 


17 


27 




11) 


Design, Cond. & 
Eval. of Exps. 


23 


13 


6 


11 


13 


35 


20 


9 


17 


20 




12) 


Computer Progr. 


18 


16 


12 


9 


11 


27 


24 


18 


14 


17 
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APPENDIX C 



USAGE FREQUENCIES BY SERVICE RANK. FREQUENCY--CAPT/MAJ = 52 

Paw Score Pero 



TECHNIQUE 



a. GENERAL OR TECHNIQUES 



1) 


Linear Prograirming 


34 


18 


0 


65 


35 


0 


2) 


Nonlinear Programming 


45 


7 


0 


87 


13 


0 


3) 


Dynamic Prograirming 


46 


5 


1 


88 


10 


2 


4) 


Integ. & Mixed Prograirming 


49 


3 


0 


94 


6 


0 


5) 


Heuristic Prograirming 


46 


5 


1 


88 


10 


2 


6) 


Bayesian Decision Analysis 


38 


14 


0 


73 


27 


0 


7) 


Networks, Flows and Graphs 


10 


41 


1 


19 


79 


2 


8) 


Risk Analysis 


31 


20 


1 


60 


38 


2 


9) 


Cost Estimation 


14 


34 


4 


27 


65 


8 


10) 


Cost Effectiveness 


12 


33 


7 


23 


63 


13 


11) 


Reliability 


19 


26 


7 


37 


50 


13 


12) 


Inventory Theory 


41 


10 


1 


79 


19 


2 


13) 


Data Analy. & Statistical 
Inference (Parametric) 


11 


28 


13 


21 


54 


25 


14) 


Data Analy. & Statistical 
Inference ( Nonpar ame trie ) 


19 


22 


11 


37 


42 


21 


15) 


Probability Theory 


13 


26 


13 


25 


50 


25 


b. OR MODELING TECHNIQUES 














1) 


Queueing Theory & Stoch. Models 


32 


19 


1 


62 


37 


' 2 


2) 


Gaining Models 


34 


16 


2 


65 


31 


4 


3) 


Canbat Models 


31 


18 


3 


60 


35 


6 


4) 


Manpower Models 


30 


17 


5 


58 


33 


10 


5) 


Eccncmetric Models 


36 


13 


3 


69 


25 


6 


6) 


Cost Models 


23 


25 


4 


44 


48 


8 


7) 


1-0 Models 


34 


18 


0 


65 


35 


0 


8) 


Forecasting 


30 


19 


3 


58 


37 


6 


c. OTHER RELATED SUBJECTS 














1) 


Calculus 


25 


23 


4 


48 


44 


8 


2) 


Linear Algebra 


30 


19 


3 


58 


37 


6 


3) 


Economics 


23 


25 


4 


44 


48 


8 


4) 


Capital Budgeting 


42 


9 


1 


81 


17 


2 


5) 


Policy Analysis 


38 


12 


2 


73 


23 


4 


6) 


Ccmbat Analysis 


28 


20 


4 


54 


38 


8 


7) 


Equipment Replacement 


25 


24 


3 


48 


46 


6 


8) 


Maintenance and Repair 


25 


23 


4 


48 


44 


8 


9) 


Transportation 


29 


23 


0 


56 


44 


0 


10) 


Human Factors 


25 


22 


5 


48 


42 


10 


ID 


Design, Cond. & Eval. of Exps. 


16 


27 


9 


31 


52 


17 


12) 


Computer Prograirming 


19 


22 


11 


37 


42 


21 
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Percentage 



Frequency — LT. Col. /Col. = 14 

Raw Score 









co 






CO 








| 


1 




| 


n 


cn 


TECHNIQUE 


1 


8 


1 


1 


§ 

CO 


l 


a. GENERAL OR TECHNIQUES 














1) 


Linear Progr arming 


9 


5 


0 


64 


36 


0 


2) 


Nonlinear Prograirming 


11 


3 


0 


79 


21 


0 


3) 


Dynamic Programming 


11 


3 


0 


79 


21 


0 


4) 


In teg. & Mixed Programming 


12 


2 


0 


86 


14 


0 


5) 


Heuristic Progr arrming 


13 


1 


0 


93 


7 


0 


6) 


Bayesian Decision Analysis 


10 


4 


0 


71 


29 


0 


7) 


Networks, Flows and Graphs 


4 


8 


2 


29 


57 


14 


8) 


Risk Analysis 


6 


6 


2 


43 


43 


14 


9) 


Cost Estimation 


4 


9 


1 


29 


64 


7 


10) 


Cost Effectiveness 


1 


10 


3 


7 


71 


21 


ID 


Reliability 


6 


7 


1 


43 


50 


7 


12) 


Inventory Theory 


11 


2 


1 


79 


14 


7 


13) 


Data Analy. & Statistical 
















Inference ( Parametric ) 


2 


10 


2 


14 


71 


14 


14) 


Data Analy. & Statistical 
















Inference ( Nonpar ame trie ) 


3 


9 


2 


21 


64 


14 


15) 


Probability Theory 


2 


11 


1 


14 


79 


7 


b. OR MODELING TECHNIQUES 














1) 


Queueing Theory & Stoch. Models 


8 


6 


0 


57 


43 


0 


2) 


Gaming Models 


7 


6 


1 


50 


43 


7 


3) 


Combat Models 


6 


7 


1 


43 


50 


' 7 


4) 


Manpower Models 


8 


5 


1 


57 


36 


7 


5) 


Econometric Models 


12 


2 


0 


86 


14 


0 


6) 


Cost Models 


8 


6 


0 


57 


43 


0 


7) 


1-0 Models 


8 


5 


1 


57 


36 


7 


8) 


Forecasting 


12 


2 


0 


86 


14 


0 


c. OTHER RELATED SUBJECTS 














1) 


Calculus 


7 


7 


0 


50 


50 


0 


2) 


Linear Algebra 


10 


4 


0 


71 


29 


0 


3) 


Economics 


10 


1 


3 


71 


7 


21 


4) 


Capital Budgeting 


7 


5 


2 


50 


36 


14 


5) 


Policy Analysis 


7 


3 


4 


50 


21 


29 


6) 


Combat Analysis 


6 


6 


2 


43 


43 


14 


7) 


Equipment Replacement 


7 


7 


0 


50 


50 


0 


8) 


Maintenance and Repair 


7 


7 


0 


50 


50 


0 


9) 


Transportation 


9 


5 


0 


64 


36 


0 


10) 


Human Factors 


6 


7 


1 


43 


50 


7 


11) 


Design, Conduct & Eval. of Exps. 


4 


8 


2 


29 


57 


14 


12) 


Computer Prograirming 


9 


4 


1 


64 


29 


7 
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APPENDIX D 



IMPORTANCE FREQUENCIES BY SERVICE RANK. Inportance--Capt/Maj =52 



Raw Score 






Percentage 



■p 

3 



TECHNIQUE 


1 


2 


3 


4 


5 


1 


2 


3 


4 


5 


a. GENERAL OR TECHNIQUES 
1) Linear Programming 


2 


5 


15 


10 


20 


4 


10 


29 


19 


38 


2) 


Nonlinear Program. 


0 


3 


6 


14 


29 


0 


6 


12 


27 


56 


3) 


Dynamic Progrartining 


0 


1 


5 


19 


27 


0 


2 


10 


37 


52 


4) 


In teg. & Mixed Prog. 


0 


1 


7 


11 


33 


0 


2 


13 


21 


63 


5) 


Heuristic Programming 


2 


1 


5 


9 


35 


4 


2 


10 


17 


67 


6) 


Bayesian Dec. Analy. 


0 


6 


11 


9 


26 


0 


12 


21 


17 


50 


7) 


Networks, Flows 
and Graphs 


4 


19 


16 


8 


5 


8 


37 


31 


15 


10 


8) 


Risk Analysis 


4 


12 


13 


9 


14 


8 


23 


25 


17 


27 


9) 


Cost Estimation 


11 


11 


13 


8 


9 


21 


21 


25 


15 


17 


10) 


Cost Effectiveness 


13 


12 


13 


8 


6 


25 


23 


25 


15 


12 


11) 


Reliability 


9 


12 


15 


7 


9 


17 


23 


29 


13 


17 


12) 


Inventory Theory 


1 


7 


8 


16 


20 


2 


13 


15 


31 


38 


13) 


Data Analy. & Stat. 
Inference (Par am. ) 


19 


14 


8 


5 


6 


37 


27 


15 


10 


12 


14) 


Data Analy. & Stat. 
Inference (Ncnparam.) 


17 


12 


6 


9 


8 


33 


23 


12 


17 


15 


15) 


Probability Theory 


20 


14 


5 


4 


9 


38 


27 


10 


8 


17 


b. OR MODELING TECHNIQUES 

1) Queueing Theory & 
Stochastic Models 


1 


10 


12 


13 


16 


2 


19 


23 


25 


31 


2) 


Gaming Models 


3 


14 


7 


9 


19 


6 


27 


13 


17 


37 


3) 


Combat Models 


7 


11 


3 


7 


24 


13 


21 


6 


13 


46 


4) 


Manpower Models 


10 


5 


9 


12 


16 


19 


10 


17 


23 


31 


5) 


Econometric Models 


2 


10 


10 


12 


18 


4 


19 


19 


23 


35 


6) 


Cost Models 


7 


15 


10 


9 


11 


13 


29 


19 


17 


21 


7) 


1-0 Models 


5 


8 


6 


9 


24 


10 


15 


12 


17 


46 


8) 


Forecasting 


6 


12 


8 


8 


18 


12 


23 


15 


15 


35 



C. OTHER RELATED SUBJECTS 



1) 


Calculus 


7 


13 


9 


9 


14 


13 


25 


17 


17 


27 


2) 


Linear Algebra 


4 


7 


19 


6 


16 


8 


13 


37 


12 


31 


3) 


Economics 


8 


14 


10 


10 


10 


15 


27 


19 


19 


19 


4) 


Capital Budgeting 


0 


9 


11 


12 


20 


0 


17 


21 


23 


38 


5) 


Policy Analysis 


2 


11 


12 


13 


14 


4 


21 


23 


25 


27 


6) 


Combat Analysis 


6 


9 


11 


4 


22 


12 


17 


21 


8 


42 


7) 


Equipment Repl. 


2 


14 


8 


12 


16 


4 


27 


15 


23 


31 


8) 


Maintenace & Repair 


4 


11 


12 


10 


15 


8 


21 


23 


19 


29 


9) 


Transportation 


1 


11 


9 


17 


14 


2 


21 


17 


33 


27 


10) 


Human Factors 


9 


10 


11 


8 


14 


17 


19 


21 


15 


27 


11) 


Des., Ccnd. &Evl. Exp. 18 


11 


5 


7 


11 


35 


21 


10 


13 


21 


12) 


Ccnputer Programming 


16 


12 


8 


7 


9 


31 


23 


15 


13 


17 
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Importance — LT Col/Col = 14 

Raw Score Percentage 



TECHNIQUE 


Extremely 

Inportant 


2 


3 


4 


Not 

<J1 

Inportant 


__ Extremely 
Inportant 


2 


3 


4 


Not 

cn 

Inportant 


a. 


GENERAL OR TECHNIQUES 
























1) 


Linear Programming 


2 


2 


3 


3 


4 


14 


14 


21 


21 


29 




2) 


Nonlinear Programming 


0 


1 


4 


4 


5 


0 


7 


29 


29 


36 




3) 


Dynamic Programming 


0 


0 


4 


5 


5 


0 


0 


29 


36 


36 




4) 


Integ. & Mixed Prog. 


0 


0 


4 


4 


6 


0 


0 


29 


29 


43 




5) 


Heuristic Programming 


0 


0 


3 


4 


7 


0 


0 


21 


29 


50 




6) 


Bayesian Dec. Analy. 


1 


1 


3 


5 


4 


7 


7 


21 


36 


29 




7) 


Networks, Flows 


























and Graphs 


1 


3 


r* 

D 


3 


2 


7 


21 


36 


21 


14 




8) 


Risk Analysis 


1 


3 


3 


2 


5 


7 


21 


21 


14 


36 




9) 


Cost Estimation 


1 


2 


4 


4 


3 


7 


14 


29 


29 


21 




10) 


Cost Effectiveness 


2 


4 


5 


2 


1 


14 


29 


36 


14 


7 




11) 


Reliability 


1 


5 


2 


5 


1 


7 


36 


14 


36 


7 




12) 


Inventory Theory 


1 


0 


2 


6 


5 


7 


9 


14 


43 


36 




13) 


Data Analy. & St at. 


























Inf. (Parametric) 


3 


3 


4 


2 


2 


21 


21 


29 


14 


14 




14) 


Data Analy. & Stat. 


























Inf. (NOnparametric) 


4 


3 


3 


2 


2 


29 


21 


21 


14 


14 




15) 


Probability Theory 


2 


5 


4 


2 


1 


14 


36 


29 


14 


7 


b. 


OR MODELING TECHNIQUES 
























1) 


Queueing Theory & 
















- 










Stochastic Models 


1 


2 


3 


5 


3 


7 


14 


21 


36 


21 




2) 


Gaming Models 


2 


2 


2 


6 


2 


14 


14 


14 


43 


14 




3) 


Combat Models 


2 


1 


3 


5 


3 


14 


7 


21 


36 


21 




4) 


Manpower Models 


2 


0 


4 


6 


2 


14 


0 


29 


43 


14 




5) 


Econometric Models 


0 


1 


2 


7 


4 


0 


7 


14 


50 


29 




6) 


Cost Models 


1 


4 


3 


5 


1 


7 


29 


21 


36 


7 




7) 


1-0 Models 


0 


3 


1 


7 


3 


0 


21 


7 


50 


21 




8) 


Forecasting 


1 


1 


3 


4 


5 


7 


7 


21 


29 


36 


c. 


OTHER RELATED SUBJECTS 
























1) 


Calculus 


1 


4 


4 


2 


3 


7 


29 


29 


14 


21 




2) 


Linear Algebra 


0 


3 


5 


3 


3 


0 


21 


36 


21 


21 




3) 


Economics 


3 


1 


3 


4 


3 


21 


7 


21 


29 


21 




4) 


Capital Budgeting 


2 


0 


3 


3 


6 


14 


0 


21 


21 


43 




5) 


Policy Analysis 


4 


0 


2 


3 


5 


29 


0 


14 


21 


36 




6) 


Combat Analysis 


3 


2 


2 


4 


3 


21 


14 


14 


29 


21 




7) 


Equipment Repl. 


0 


2 


3 


4 


5 


0 


14 


21 


29 


36 




8) 


Maintenance & Repair 


0 


1 


3 


6 


4 


0 


7 


21 


43 


29 




9) 


Transportation 


0 


2 


2 


4 


6 


0 


14 


14 


29 


43 




10) 


Human Factors 


2 


3 


2 


3 


4 


14 


21 


14 


21 


29 




ID 


Des. , Ccnd. &Evl. Exp. 


5 


2 


1 


4 


2 


36 


14 


7 


29 


14 




12) 


Computer Programming 


2 


4 


4 


2 


2 


14 


29 


29 


14 


14 
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Inport ant 



APPENDIX E 



EXAMPLE OF AN OPERATIONS ANALYSIS 
CURRICULUM FOR U.S.M.C. STUDENTS 
ATTENDING THE NAVAL POSTGRADUATE 
SCHOOL 



First Quarter 



Introductory Computing for Operations Analysis 

Topics in Calculus 

Probability 

History & Nature of Operations Analysis 

Linear Algebra 

Seminar for OA students 

Second Quarter 



Linear Programming 

Selected Topics from Advanced Calculus 
Probability and Statistics 
Introduction to Mathematical Economics 
Seminar for OA students 

Third Quarter 



Statistics 
Stochastic Models I 
Computational Methods for OR 
Networks, Flows and Graphs 
Seminar for OA students 

Fourth Quarter 



Utility Theory & Resource Allocation 
Human Factors in System Design I 
Analysis of Operational Data 
Systems Simulation 
Seminar for OA students 

Fifth Quarter (includes 6 week Experience Tour) 

Systems Analysis I 
Combat Models & Wargaming 
Seminar for OA students 

Sixth Quarter 



Non-Linear and Dynamic Programming 
Stochastic Models II 
Design of Experiments 
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Elective 

Seminar for OA students 
Seventh Quarter 



Thesis Research 

Reliability and Weapon System Effectiveness 

Evaluation of Human Factors Data 

Elective 

Seminar for OA students 
Eighth Quarter 



Thesis Research 
Cost Estimation 
Elective 
Elective 

Seminar for OA students 
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